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Abstract 

We  examine  a  well  known  confidentiality  requirement  called  noninterference  and  argue  that  many 
systems  do  not  meet  this  requirement  despite  maintaining  the  privacy  of  its  users.  We  discuss 
a  weaker  requirement  called  incident-insensitive  noninterference  that  captures  why  these  systems 
maintain  the  privacy  of  its  users  while  possibly  not  satisfying  noninterference.  We  extend  this 
requirement  to  depend  on  dynamic  information  in  a  novel  way.  Lastly,  we  present  a  method 
based  on  model  checking  to  extract  from  program  source  code  the  dynamic  incident-insensitive 
noninterference  policy  that  the  given  program  obeys. 
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1  Introduction 


Given  a  multi-user  system,  a  user  might  wonder  how  it  protects  his  privacy.  Such  a  user  would 
benefit  from  a  summary  of  who  else  may  use  the  system  to  access  his  information  and  under  what 
conditions.  We  hope  to  develop  a  tool  that  automatically  produces  such  a  summery,  or  dynamic 
confidentiality  policy,  from  the  source  code  of  the  program  controlling  such  a  system.  Before  we  may 
describe  an  approach  to  this  problem,  we  must  first  consider  what  it  means  for  a  user’s  information 
to  remain  confidential. 

Confidentiality  Requirements.  What  must  a  system  keep  secret  to  maintain  the  privacy  of  its 
users?  No  single  answer  is  correct  for  all  systems:  different  balances  of  privacy  and  functionality 
result  in  systems  with  different  confidentiality  guarantees. 

Consider  a  system  with  a  high-level  user  H  and  a  low-level  user  L,  whom  H  does  not  trust. 
The  user  H  desires  that  the  system  guarantees  that  the  user  L  has  no  way  of  learning  about  the 
inputs  of  H  to  the  system.  This  guarantee  may  be  formalized  as  a  confidentiality  assertion.  Such 
a  formalization  must  make  clear  what  exactly  it  means  for  the  untrusted  user  L  to  learn  about  an 
input  of  H.  Each  different  formalization  of  this  concept  corresponds  to  a  different  confidentiality 
requirement. 

One  of  the  most  well  known  and  earliest  confidentiality  requirements  is  noninterference  as 
defined  by  Goguen  and  Meseguer  [7]  and  later  extended  to  nondeterministic  systems  by  McCul¬ 
lough  [20,  21].  Informally,  the  confidentiality  assertion  that  the  user  H  is  noninterfering  with  the 
user  L  requires  that  the  set  of  possible  outputs  seen  by  L  is  the  same  regardless  of  any  inputs 
provided  by  H  to  the  system.  This  requirement  is  so  strong  that  the  user  L  may  not  even  know  if 
H  has  provided  any  inputs  to  the  system. 

Such  a  strong  requirement  is  often  too  stringent,  that  is,  it  places  so  much  emphasis  on  privacy 
that  it  prevents  some  systems  from  achieving  a  reasonable  level  of  functionality.  In  many  realistic 
systems,  allowing  the  user  L  to  know  that  the  user  H  has  entered  an  input  into  the  system  is  accept¬ 
able  as  long  as  L  does  not  learn  about  the  contents  of  the  input.  In  Section  3,  we  provide  examples 
of  such  systems  and  present  a  weakened  form  of  noninterference  that  allows  L  to  learn  that  H  has 
provided  inputs  to  the  system  while  still  protecting  the  contents  of  these  inputs.  We  also  formalize 
a  weakened  confidentiality  requirement  based  on  this  observation  that  we  call  incident-insensitive 
noninterference  since  the  user  L  is  allowed  to  learn  of  the  incident  of  the  input.  Likewise,  we  call  the 
original  noninterference  requirement  of  Goguen  and  Meseguer  incident- sensitive  noninterference. 

Dynamic  Confidentiality  Assertions.  The  confidentiality  assertions  described  thus  far  have 
been  static:  they  hold  between  two  users  regardless  of  their  actions.  Often  static  requirements 
cannot  capture  the  confidentiality  guarantee  that  a  system  should  make  to  its  users.  For  example, 
consider  a  system  that  stores  emails  for  its  users.  The  system  should  not  allow  a  user  to  read  any 
of  the  emails  unless  that  user  provides  the  correct  password.  To  formally  capture  such  a  guarantee 
requires  a  dynamic  confidentiality  assertion,  an  assertion  that  some  confidentiality  requirement 
should  hold  between  two  users  unless  some  condition  that  depends  on  dynamic  information  is  met 
at  runtime. 

Along  with  noninterference,  Goguen  and  Meseguer  introduced  a  form  of  dynamic  confidentiality 
assertion  [7].  A  dynamic  assertion  of  their  form  declares  that  an  input  from  a  high-level  user  H 
should  remain  unknown  to  a  low-level  user  L  unless  some  predicate  holds  of  the  inputs  that  preceded 
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the  input  in  question.  Since  the  dynamic  assertion  may  only  depend  on  the  inputs  that  precede 
the  input  in  question,  we  term  their  formulation  at-input- checking. 

In  the  password  example  above,  the  dynamic  assertion  should  hold  unless  the  user  L  enters  the 
correct  password,  an  event  that  might  occur  after  the  system  has  already  received  an  email  (from 
H).  Since  at-input-checking  assertions  may  depend  only  on  inputs  received  before  such  an  email 
arrived,  they  cannot  capture  the  needed  assertion. 

To  fix  this  problem,  we  remove  the  requirement  that  the  predicate  of  a  dynamic  assertion  may 
only  depend  on  inputs  that  precede  the  input  in  question.  Under  our  formulation,  a  dynamic 
assertion  will  require  that  an  input  be  protected  until  enough  dynamic  information  is  collected 
to  rule  otherwise.  This  information  may  come  at  anytime  as  long  the  input  in  question  does  not 
affect  any  outputs  to  the  user  L  until  it  arrives.  If  such  input  never  comes,  the  input  will  always 
be  protected.  We  term  this  formulation  at- output- checking  since  at  the  time  of  an  output,  all  the 
inputs  that  have  arrived  may  affect  whether  the  output  may  depend  on  some  previous  input,  rather 
than  just  those  inputs  that  preceded  the  input  in  question.  In  Section  4,  we  formalize  this  new 
form  of  dynamic  assertion. 

Policy  Extraction.  A  set  of  dynamic  incident-insensitive  noninterference  (DIINI)  assertions 
defines  a  DIINI  policy.  Given  a  DIINI  policy,  a  programmer  can  take  two  different  approaches 
to  ensuring  that  a  program  obeys  the  policy.  In  the  first  approach,  the  programmer  codes  with 
the  policy  in  mind  and  manually  inserts  any  dynamic  checks  that  the  program  must  perform  to 
ensure  that  the  policy  is  obeyed.  In  the  second  approach,  the  programmer  abstracts  the  policy 
enforcement  mechanism  from  core  application  logic  of  the  program  and  configures  the  program 
with  an  explicit  representation  of  the  policy. 

While  the  first  approach  is  usually  easier  to  implement,  the  second  approach  has  many  advan¬ 
tages.  Firstly,  an  organization  with  an  explicit  policy  may  apply  that  policy  to  multiple  programs. 
Secondly,  the  decoupling  of  policy  from  application  logic  allows  multiple  organizations  with  differ¬ 
ing  confidentiality  policies  to  use  a  single  program  since  each  organization  may  separately  configure 
the  program  to  enforce  its  policy.  Thirdly,  having  a  centralized  policy  facilitates  reasoning  about 
the  policy  and  editing  it. 

To  gain  these  advantages  for  legacy  programs  written  using  the  first  approach,  the  program 
maintainers  should  convert  them  to  use  a  explicit  policy  as  in  the  second  approach.  A  tool  that 
aggregates  the  manually  inserted  dynamic  checks  used  to  ensure  that  the  program  obeys  the  policy 
together  into  an  explicit  representation  of  this  policy  would  ease  this  conversion,  especially  for  large 
programs. 

Many  other  uses  for  such  a  tool  exist.  A  system  administrator  could  examine  the  extracted 
policy  by  hand  or  use  tools  to  answer  queries  about  the  policy.  Furthermore,  such  a  tool  could 
verify  that  an  extracted  policy  meets  the  requirements  of  a  specified  policy.  Even  in  the  absence  of 
a  formal  specification,  change-impact  analysis  is  possible:  given  application  code  before  and  after 
some  set  of  edits,  one  could  compare  the  extracted  policies  to  ensure  that  the  program  edits  has 
introduced  no  new  security  holes. 

In  Section  5,  we  present  an  approach  based  on  model  checking  for  this  policy  extraction  problem. 
Our  approach  tracks  the  flow  of  information  through  the  program  in  a  manner  similar  to  type 
systems  that  track  information  flow  [29].  However,  our  approach  allows  the  same  variable  to  carry 
both  high-  and  low-level  information  without  the  low-level  information  being  considered  high- 
level  preventing  an  overly  conservative  analysis.  Furthermore,  our  approach  attempts  to  rule  out 
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infeasible  paths.  While  these  features  matter  little  in  the  context  of  writing  a  program  with  type 
analysis  in  mind,  they  become  important  in  our  primary  use  case  of  extracting  policies  from  legacy 
code. 

Road  Map  and  Contributions.  The  order  of  this  paper  mirrors  the  development  of  this  in¬ 
troduction:  After  handling  some  technical  preliminaries  in  Section  2,  we  motivate  and  present 
incident-insensitive  noninterference  in  Section  3.  Then  we  present  our  formulation  of  dynamic 
confidentiality  assertions  in  Section  4.  With  the  notation  of  dynamic  confidentiality  policy  fully 
formalized,  we  at  last  return  to  the  original  motivation  of  this  work,  automated  policy  extraction, 
in  Section  5.  Lastly,  we  cover  related  work. 

The  three  main  sections  of  this  work  each  represent  a  separate  contribution: 

•  Section  3  motives  the  need  for  incident-insensitive  noninterference  clarifies  its  relation  to 
the  original  definition  of  noninterference.  (Since  a  similar  confidentiality  requirement  has 
appeared  in  the  literature  before  [25],  we  do  not  consider  the  presentation  of  the  requirement 
to  be  our  contribution  per  se.) 

•  Section  4  motives  the  need  for  and  presents  a  more  general  notation  of  dynamic  confidentiality 
assertion,  which  allows  for  the  expression  of  realistic  policies. 

•  Section  5  provides  an  approach  to  automated  policy  extraction. 

An  additional  contribution  is  that  of  unwinding  conditions  for  incident-insensitive  noninterference 
in  both  its  static  and  dynamic  form.  Using  unwinding  conditions  eases  proving  that  a  system 
satisfies  a  noninterference  policy.  We  demonstrate  their  usefulness  by  employing  them  to  prove  the 
correctness  of  our  approach  to  policy  extraction. 

2  The  System  Model 

Automata.  The  input-output  behavior  of  a  system  determines  what  confidentiality  assertions 
it  satisfies.  Agents  acting  in  various  security  domains  create  the  inputs  and  receive  the  outputs. 
Each  domain  is  a  different  entity  or  class  of  entities  that  might  interact  with  the  system.  For 
example,  the  domains  might  be  Top  Secret,  Secret,  Classified,  and  Unclassified  for  modeling  the  flow 
of  information  between  security  classes  in  a  military  system  and  if  the  actual  identity  of  the  entity 
is  irrelevant  (only  its  security  clearance  matters).  For  modeling  the  use  of  different  resources,  the 
domains  might  be  Hard  Drive,  Network,  and  User. 

These  domains  interact  with  one  another  by  using  the  system.  We  will  model  such  a  system  as 
an  automaton.  Formally,  an  system  automaton  m  consists  of 

•  a  set  of  inputs  /, 

•  a  set  of  outputs  O  such  that  /  fl  O  =  0, 

•  a  set  of  domains  D, 

•  a  function  dom  :  A  — >  D  that  assigns  to  each  action  the  domain  that  created  or  received  it 
where  the  set  of  actions  A  is  I  U  O, 

•  a  set  of  states  Q, 
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•  a  start  state  go  £  Q,  and 

•  a  transition  relation  ->•  C  Q  x  A  x  Q. 

We  write  q\  ->  q2  if  (qi,  a,  q-i)  £  We  write  q\  ->  q-2  for  a  £  A*  if  either 

•  a  =  []  and  qi  =  q2,  or 

•  a  =  a:a',  q\  A  q[,  and  q[  q2 

where  []  is  the  empty  sequence  and  a:a  is  the  sequence  formed  by  prepending  a  to  a.  (For  example, 
a:b:[]  =  [a,  b].)  Since  we  never  have  a  list  of  lists,  we  abuse  notation  and  also  use  :  to  append  lists 
and  to  add  elements  to  their  end. 

The  above  automaton  model  is  asynchronous  and  nondeterministic,  which  greatly  complicates 
proofs  about  them.  We  use  asynchronous  automata  since  programs  often  produce  output  for  one 
user  without  producing  it  for  other  users.  We  require  nondeterminism  since  we  have  the  end  goal 
of  model  checking  in  mind  and  model  checking  works  over  a  nondeterministic  abstraction  of  an 
actual  program. 


Behaviors.  Let  the  set  of  behaviors  of  an  automaton  m  =  (/,  O,  D,  dom,  Q,  qo,  -*)  be 

behv(m)  =  {a  £  A*  \3q  £  Q  s.t.  qo~>  q} 


Each  behavior  represents  one  way  in  which  the  system  might  operate.  Since  each  domain  has 
control  over  its  input  actions,  each  domain  may  affect  which  behaviors  the  system  can  execute.  Let 
l  £  I*  represent  a  sequence  of  inputs.  If  the  system  is  subjected  to  the  inputs  of  1  and  no  other 
inputs,  then  the  system  may  only  execute  those  behaviors  that  include  all  the  inputs  of  t  in  order 
and  no  other  inputs. 

To  formalize  this  notion,  let  us  first  define  the  restrict  function  |_-J-  :  A*  x  2A  — >  A*.  The 
restrict  function  takes  a  sequence  a  and  a  subset  A1  of  A  and  returns  the  sequence  \ol\a'  which 
only  includes  the  elements  of  a  that  are  in  A! .  Let  |_ckJa'  be  defined  as  follows: 


LDJa' 


[a:a\Af 


0 

j a:([a\A')  if  a  £  A' 

1  L«Ja'  otherwise 


where  A'  £2A.  For  example,  [[a,  c,  a]J  {a  b}  =  [a,  a]  and  |_[c,  a,  b,  c,  a] J  {a  b|  =  [a,  b,  a]. 

The  set  of  behaviors  that  are  possible  given  a  sequence  that  provides  all  the  inputs  to  the  system 
m  is  given  by  runs  :  I*  — >  2A  where 


runs(i)  =  {a  £  behv(m)  |  [a\i  =  t} 

A  domain  d  cannot  observe  all  the  actions  of  a  system:  d  can  only  observe  those  actions  a  such 
that  dom  (a)  =  d.  Thus,  if  the  system  executes  a  behavior  a ,  then  the  domain  d  only  sees  the 
sequence  of  actions  [a\j^d  where  Ad  =  {  a  £  A  \  dom(a)  =  d  }.  If  two  behaviors  a±  and  02  are  such 
that  \a\\Ad  =  L*-T2 J Ad >  then  cq  and  a 2  provide  the  domain  d  with  the  same  observations  and  thus 
look  the  same  to  domain  d.  In  general,  if  a  domain  d  sees  the  action  sequence  a,  d  will  only  be 
able  to  tell  that  some  behavior  a'  such  that  |_a/_lAd  =  a  was  executed;  d  will  not  know  which  one. 
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Let  us  raise  |_-J  ■  to  work  over  sets  of  sequences  as  follows:  |_{cki  ,  «2, . .  .}J  a'  =  {|_qiJa'!  Lq2Ja'>---}- 
Then,  if  for  two  input  sequences  L\  and  1.2  of  a  system  m.  |_runs(ii)J  =  [runs^)]^  then  domain 
d  cannot  tell  between  when  L\  or  1,2  is  the  input  sequence  to  in.  This  provides  an  opportunity  to 
prevent  a  domain  from  learning  the  inputs  of  another  domain. 

Adding  Internal  Transitions.  A  problem  with  the  above  automation  model  is  that  each  tran¬ 
sition  either  results  in  output  or  is  the  result  of  input.  This  limitation  does  not  allow  internal 
transitions.  To  allow  internal  transitions,  we  allow  a  distinguished  action  r  ^  A  that  represents  an 
action  that  no  domain  can  observe.  If  the  users  may  deduce  the  execution  of  an  internal  transition 
(perhaps  by  timing),  then  this  model  is  inappropriate. 

Since  users  cannot  observe  internal  transitions,  they  should  not  show  up  in  the  behaviors  of 
a  system  and  we  must  redefine  behv  with  this  in  mind.  Let  q — >q'  iff  q^>q',  or  both  q-^-q"  and 
q"  —>  q'  for  some  q"  Let 

behv(m)  =  {  a  G  A*  \  3q  €  Q  s.t.  qo  q  } 
where  — >  is  raised  to  sequences  a  in  the  same  manner  as  was. 

3  Noninterference 

3.1  What  is  Confidentiality? 

Consider  the  following  simple  program: 

bool  in  =  load("secret-f ile .db") ; 
print ( ‘x’ ) ; 

The  first  line  reads  in  the  contents  of  a  secret  file.  The  second  line  simply  prints  the  character  ‘x’  to 
the  low-level  user.  If  we  model  the  reading  of  the  secret  file  as  receiving  input  from  a  high-level  user, 
then  this  program  fails  to  meet  the  requirements  of  incident-sensitive  noninterference  as  defined 
by  Goguen  and  Meseguer  [7].  The  reason  is  that  the  low-level  user  does  not  see  the  output  ‘x’ 
unless  the  high-level  user  produces  input,  which  allows  the  load  statement  to  stop  blocking  and 
terminate.  Thus,  the  low-level  user  has  learned  that  the  high-level  user  has  interacted  with  the 
system.  This  violation  occurs  even  though  the  low-level  user  clearly  does  not  learn  anything  about 
the  contents  of  secret-file .  db.  (We  formalize  this  example  in  Appendix  A.l.) 

We  believe  that  in  many  cases  allowing  the  low-level  user  to  know  that  the  high-level  user  is 
interacting  with  the  system  is  acceptable  as  long  as  the  low-level  user  does  not  learn  the  contents 
of  these  interactions.  Consider  the  following  realistic  examples: 

•  “Upon  startup,  a  web  server  for  online  banking  receives  financial  records  from  a  secure  database 
before  answering  any  queries  from  users.” 

This  web  server  violates  incident-sensitive  noninterference  since  if  the  web  server  answers  the 
user’s  queries  with  low-level  outputs,  the  user  will  know  that  the  server  has  consumed  high-level 
input  from  the  database.  This  violation  holds  even  if  the  inputs  consumed  from  the  high-level 
database  did  not  influence  the  server’s  response  to  the  low-level  user.  However,  such  a  system 
maintains  an  acceptable  level  of  confidentiality  since  the  low-level  user  cannot  learn  what  inputs 
the  high-level  database  provided  to  the  server  and  the  low-level  user  learning  that  server  has 
received  high-level  input  only  tells  the  low-level  user  that  system  is  working  correctly. 
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•  “A  student  is  applying  for  graduate  school  online.  During  the  application  process,  both  the  stu¬ 
dent  and  the  professors  recommending  him  must  enter  information  into  the  application  database. 
Once  the  recommending  professors  have  finished,  the  student  receives  a  notice  stating  that  the 
graduate  school  has  received  his  recommendations.  The  applicant  is  not  allowed  access  to  his 
recommendation.” 

The  low-level  student  only  receives  the  notice  if  the  professors  have  entered  their  high-level 
recommendations.  Thus,  by  receiving  the  notice,  the  student  learns  that  the  system  has  consumed 
high-level  inputs.  This  violates  incident-sensitive  noninterference  even  if  the  content  of  the  high- 
level  recommendation  does  not  affect  the  content  of  the  notice. 

•  “PhoneBook  is  a  system  produced  by  NS  that  organizes  phone  numbers  for  a  law  firm.  While 
adding  a  new  contact,  PhoneBook  reaches  an  error  state.  PhoneBook  offers  to  send  a  bug  report 
to  NS  stating  only  that  the  system  failed  to  add  a  new  contact.  The  law  firm  considers  any 
personally  identifiable  information  about  its  contacts  to  be  private.” 

Since  the  error  state  was  reached  during  the  addition  of  contact  information,  the  bug  report 
indicates  that  the  system  was  receiving  high-level  contact  information.  Thus,  even  if  the  bug 
report  maintains  the  privacy  of  the  contacts  by  not  providing  any  information  about  them,  the 
system  will  still  violate  incident-sensitive  noninterference  by  sending  the  bug  report  to  NS,  which 
is  low-level. 

•  “A  physician  uses  a  computer  to  record  his  interactions  with  patients.  The  physician  enters 
into  the  computer  both  the  treatment  rendered  and  the  fee  charged  (the  physician  negotiates 
the  fee  with  each  patient).  The  system  should  only  allow  the  physician  to  access  the  treatment. 
However,  the  system  provides  the  fee  to  his  secretary  for  billing.” 

Since  the  low-level  secretary  receives  a  notice  to  bill  a  patient  from  the  system,  he  knows  that 
the  physician  has  entered  into  the  system  a  high-level  input  describing  the  treatment.  This 
knowledge  implies  a  violation  of  incident-sensitive  noninterference  even  if  the  notice  does  not 
reveal  any  information  about  the  treatment. 

From  these  examples,  it  should  be  clear  that  often  simply  learning  that  some  high-level  input 
has  taken  place  does  not  provide  the  low-level  user  with  enough  information  to  constitute  a  vi¬ 
olation  of  the  high-level  user’s  confidentiality.  However,  most  confidentiality  requirements  (e.g., 
restrictiveness  [20,  21,  22]  and  separability  [23])  are  incident  sensitive :  they  prohibit  low-level  users 
from  learning  that  any  high-level  input  has  taken  place. 

What  we  desire  are  incident-insensitive  requirements,  ones  that  allow  low-level  users  to  learn 
that  high-level  input  has  taken  place  while  protecting  the  contents  of  these  high-level  inputs.  Intu¬ 
itively,  a  system  obeys  incident-insensitive  noninterference  if  the  content  of  inputs  from  a  high-level 
user  has  no  effect  on  the  outputs  that  a  low-level  user  sees.  To  make  this  slightly  more  formal, 
incident-insensitive  noninterference  requires  that  the  set  of  possible  outputs  seen  by  a  low-level  user 
is  the  same  regardless  of  the  content  of  the  inputs  from  high-level  users.  Note  that  the  low-level 
user  is,  however,  allowed  to  learn  that  the  high-level  user  sent  inputs  to  the  system. 

Incident-insensitive  requirements  have  appeared  in  works  on  information-flow  type  systems 
(Sabelfeld  and  Myers  provide  a  survey  [29]).  O’Neill  et  al.  have  proved  that  these  type  systems 
ensure  that  a  program  obeys  an  incident-insensitive  requirement  they  simply  call  “noninterfer¬ 
ence”  [25]. 
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The  rest  of  this  section  formalizes  a  slightly  weaker  form  of  O’Neill’s  noninterference.  We  delay 
describing  how  our  formulation  is  weaker  than  O’Neill’s  until  Section  6. 

3.2  Noninterference  Formalized 

First  we  present  policies  in  general.  Then  we  present  the  statement  of  incident-sensitive  noninterfer¬ 
ence  as  defined  by  McCullough  for  nondeterministic  systems  [20,  21].  After  showing  our  definition 
for  incident-insensitive  noninterference,  we  compare  the  two. 


Policies.  For  a  system  m,  a  generic  confidentiality  policy  is  an  reflexive,  transitive  relation  on 
D.  We  write  df  -/»  dt  iff  -<(df  ^  dt).  If  df  dt,  then  information  about  df  should  not  flow  to  dt. 
A  generic  policy  does  not  specify  exactly  what  it  means  for  information  to  flow.  That  is,  a  generic 
policy  does  not  specify  a  confidentiality  requirement. 

Below,  we  formalize  two  confidentiality  requirements  that  can  give  a  generic  policy  meaning: 
incident-sensitive  noninterference  and  incident-insensitive  noninterference.  Since  these  two  require¬ 
ments  may  be  viewed  as  two  different  interpretations  that  one  may  assign  to  a  generic  policy,  we 
represent  policies  of  either  type  using  as  with  generic  policies  and  let  the  surrounding  text  make 
clear  which  type  of  policy  it  is. 


Incident-Sensitive  Noninterference.  Let  =  ff4  be  a  relation  on  input  sequences  such  that 
0  =74  [],  and  *i:ti  =74  i2u 2  iff 

•  ii  =  i2  and  i\  =74  i2, 

•  dom(ii)  -/>  d  and  l\  =74  i2'-i2,  or 

•  dom(i2)  -/>  d  and  i\:i\  =7  4  t2- 

A  system  m  obeys  as  an  incident-sensitive  noninterference  policy  iff  for  all  d  €  D  and 
/.  1./.-2  f  i% 

L\  t2  implies  [runs(ii)JAd  C  [runs(t2)JJ4<i 

Intuitively,  this  definition  says  that  if  i\  has  been  received  by  the  system  and  d  should  not  be  able 
to  rule  out  the  possibility  that  it  was  i2  that  the  system  received,  then  there  must  exist  no  behavior 
of  the  system  under  i\  that  is  impossible  under  l2  from  the  perspective  of  d. 

Incident-Insensitive  Noninterference.  Let  =^f'd  be  a  relation  on  input  sequences  such  that 
0  =7 4  0  and  hm  =7 4  7^2  iff 

•  dom(ii)  =  dom(i2), 

•  dom(ii)  d  implies  i\  =  i2,  and 


A  system  m  obeys  a  policy 
and  ti,  i2  e  /*, 

—II 


as  an  incident-insensitive  noninterference  policy  iff  for  all  d  €  D 
i2  implies  [runs(ii)J Ad  C  [runs(/.2)JJ4d 
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Comparison.  Note  that  for  all  d  and^->,  both  =^’d  and  =f'd  are  equivalence  relations.  They  are 
also  alike  in  that  if  dom(ii)  ^  d  and  dom(i2)  d,  both  require  that  i\  =  'i2  for  i\\L\  i2u2  or 
=ff’d  i2:t2  to  hold.  However,  if  dom(i1)  qU  d,  then  =^'d  still  requires  that  dom(ii)  =  dom(i2) 
whereas  =ff,d  makes  no  requirements  at  all  and  simply  drops  i\  from  consideration.  This  difference 
is  the  difference  between  incident-sensitive  noninterference  and  incident-insensitive  noninterference. 

Since  =  ff'd  places  more  requirements  on  i\  than  —ff’d,  it  should  come  as  no  surprise  that 
i\  =ff'd  12  implies  i\  =ff'd  12  (see  Lemma  3  in  Appendix  A. 2).  A  direct  result  of  this  follows: 

Theorem  1.  If  a  system  obeys  a  generic  policy  as  an  incident- sensitive  noninterference  policy, 
then  it  will  obey  as  an  incident-insensitive  noninterference  policy;  the  converse  is  not  true. 

Appendix  A. 2  provides  a  proof. 

A  specification  may  place  both  an  incident-insensitive  noninterference  policy  and  an  incident- 
sensitive  noninterference  policy  on  the  same  system.  A  specification  might  require  that  some 
users  be  incident-sensitively  noninterfering  with  a  second  group  of  users  and  incident-insensitively 
noninterfering  with  a  third  group.  The  above  theorem  makes  clear  the  relationship  between  these 
two  policies. 

3.3  Unwinding 

Since  noninterference  is  a  global  property,  proving  that  a  nontrivial  system  obeys  a  given  policy  is 

a  daunting  task.  Thus,  Goguen  and  Meseguer  provided  a  property,  the  existence  of  an  unwinding 

relation ,  to  ease  this  task  [8].  We  provide  such  a  property  for  incident-insensitive  noninterference. 

Let  q  q'  iff 
d 

a  , 

•  <7^9; 

•  q^q"  and  q"  q’;  or 

d 

•  there  exists  o  G  O  such  that  dom(o)  7^  d,  q^>  q",  and  q"  —>  q' . 

d 

Informally,  q  — U  q'  means  that  the  automaton  can  transition  from  q  to  q'  by  using  only  internal 
d 

transitions,  transitions  that  produce  output  for  a  domain  other  than  d,  and  finally  one  transition 
using  a. 

Given  a  system  automaton,  let  a  view  partition  be  a  function  from  a  domain  to  an  equivalence 
relation  on  states.  That  is,  a  view  partition  is  in  D  — >  2^x<^.  We  will  write  q\  ~  g2  if  for  the  domain 
d ,  the  states  q\  and  g2  are  within  the  relation. 

Let  a  view  partitioning  for  a  program  automaton  m  and  policy  be  called  an  incident- 
insensitive  unwinding  relation  if  it  satisfies  the  following  unwinding  conditions: 

1.  Local  Respect:  for  all  d  G  D,  q,q[  G  Q,  and  *i,i2  G  I,  if  dom(ii)  =  dom(i2),  dom(ii)  -/>  d  and 

q  — G.  q'{ ,  then  there  must  exist  q'0  G  Q  such  that  q  — G  q'2  and  q[  ~  q'2  ■ 
d  d 

2.  Step  Consistency:  for  all  d  G  D,  q\ ,  q\ ,  g2  G  Q,  and  i  G  /,  if  q\  ~g2  and  q\  then  there 

d 

must  exist  q2  G  Q  such  that  g2  — —>  q2  and  q\  ^  q2  ■ 


3.  Output  Consistency:  for  all  d  £  D,  q\ ,  q\ .  q2  G  Q ,  and  o  6  O,  if  dom(o)  =  d,  qi^q-2,  and 

q\  — —>  q[ ,  then  there  must  exist  q2  €  Q  such  that  q-2  q2  and  q[  ~  q'2  ■ 
d  "  d 

The  above  unwinding  conditions  are  much  more  complex  than  the  standard  ones  presented  for 
incident-sensitive  noninterference.  However,  incident-insensitivity  is  not  blame:  this  actually  stems 
from  using  asynchronous,  nondeterministic  automata  for  our  system  model  instead  of  synchronous, 
deterministic  automata. 

Theorem  2.  If  there  exists  an  incident-insensitive  unwinding  relation  for  a  incident-insensitive 
noninterference  policy  given  an  automaton,  then  that  automaton  obeys  the  policy. 

Appendix  A. 3  offers  the  proof. 


4  Dynamic  Policies 

4.1  Motivation 

Now  we  motivate  the  need  for  dynamic  confidentiality  assertions  by  relating  in  more  detail  the 
email  server  example  from  the  introduction.  As  described  before,  the  server  should  only  allow 
access  to  the  emails  if  the  user  supplies  the  correct  password.  The  following  program  written  in  a 
C-like  language  enforces  this  requirement: 

emails  =  load("mbox") ; 
real_pw  =  load("password") ; 
given_pw  =  read() ; 
if (given_pw  ==  real_pw) 
print (emails) ; 
else 

print ("wrong") ; 

where  the  file  "mbox"  holds  the  emails  and  "password"  holds  the  correct  password. 

To  model  this  program,  let  the  emails  be  represented  by  the  domain  e,  the  password  by  the 
domain  p,  and  user  by  the  domain  u.  Since  the  user  u  can  gain  access  to  the  emails  e  by  entering  the 
correct  password,  the  system  does  not  obey  any  policy  such  that  e^u.  However,  such  a  static 
policy  fails  to  convey  the  design  goal  of  only  allowing  the  user  access  to  the  emails  if  he  provides  the 
correct  password.  We  desire  a  policy  that  captures  how  supplying  the  correct  password  at  runtime 
changes  the  allowed  information  flows. 

To  address  such  concerns,  Goguen  and  Meseguer  presented  a  dynamic  version  of  incident- 
sensitive  noninterference  [7].  Informally,  it  allows  an  input  from  a  high-level  domain  to  be  treated 
as  insecure  (accessible  to  the  low-level  domain)  if  the  inputs  that  precede  it  satisfy  some  predicate. 
This  allows  the  security  of  an  input  to  depend  on  the  inputs  provided  before  it  at  runtime.  Since 
all  the  information  on  which  the  security  of  an  input  may  depend  is  present  at  the  time  that  the 
input  enters  the  system,  we  call  their  formulation  at-input- checking. 

The  inability  of  at-input-checking  to  consider  information  that  follows  the  input  in  question 
limits  the  expressiveness  of  at-input-checking.  In  the  above  example,  the  emails  were  the  first 
input  to  the  system.  Since  no  input  precedes  the  emails  and  the  security  of  an  input  may  only 
depend  on  those  inputs  that  precede  the  input  in  question,  the  emails  must  either  always  be  secure 
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or  always  be  insecure.  This  has  the  same  problem  as  static  policies:  we  cannot  have  the  emails  be 
secure  in  some  behaviors  of  the  system  and  insecure  in  others. 

To  fix  this  problem,  we  must  allow  the  security  of  an  input  to  depend  on  inputs  that  arrive 
after  it.  In  this  case,  the  security  of  the  emails  is  undetermined  until  the  user  has  entered  his  input. 
It  may  seem  that  such  information  comes  too  late:  How  can  information  from  the  future  be  used 
to  determine  the  security  of  an  input  now?  The  answer  is  that  the  determination  need  not  be 
made  when  the  input  has  just  arrived:  as  long  as  the  input  is  treated  as  though  it  is  secure  until 
information  becomes  available  indicating  otherwise,  this  determination  may  be  delayed. 

To  make  use  of  this  observation,  we  define  a  new  version  of  dynamic  policy  that  depends  not 
only  on  the  inputs  that  precede  the  input  in  question,  but  also  those  inputs  that  follow  it.  At  the 
time  of  an  output,  whether  that  output  may  provide  information  about  an  input  depends  on  all 
the  inputs  that  precede  that  output,  not  just  those  the  precede  the  input  in  question.  Thus,  we 
call  our  formulation  at- output- checking. 

4.2  Formalization 

Dynamic  Policies.  Let  a  generic  dynamic  policy  be  a  function  from  an  input  sequence  to  a 
static  generic  policy  (a  relation  on  domains).  Give  the  set  of  inputs  I  and  domains  D,  the  set  of 
possible  generic  dynamic  policies  is  I*  — >  2DxD .  Given  a  dynamic  policy  we  write  df  dt  if  t 
is  mapped  to  a  policy  that  allows  information  to  flow  from  df  to  dt. 

At-Input-Checking.  To  define  dynamic  incident-insensitive  noninterference  (DIINI)  using  at- 
input-checking,  we  must  replace  the  relation  =n.  Since  the  security  of  an  input  may  only  depend 
on  the  inputs  that  precede  it,  we  define  a  new  relation  =DII  that  effectively  forgets  the  inputs  that 
follow  the  input  currently  in  question.  To  achieve  this,  we  define  =DII  to  work  from  the  end  of  input 
sequences  to  their  front  forgetting  the  inputs  seen  along  the  way. 

Let  i\:i\  iff  dom(ii)  =  dom(*2),  dom(H)  d  implies  i\  =  12,  and  i\  =mi'd  L‘i-  Also 

let  []  —  dii^  []• 

A  system  m  obeys  a  DIINI  policy  ^  using  at-input-checking  iff  for  all  d  G  D  and  l\,  12  G  I*, 

=mi1,d  12  implies  |_runs(ii)J Ad  C  |_runs(?2)J 

At-Output-Checking.  Since  DIINI  using  at-output-checking  does  not  need  to  forget  any  infor¬ 
mation,  its  definition  is  actually  simpler.  We  provide  the  dynamic  policy  with  the  current  input 
sequence  L\  to  obtain  the  static  policy  '^tl  for  use  with  =n. 

A  system  m  obeys  a  DIINI  policy  ^  using  at-output-checking  iff  for  all  d  G  D  and  i.\ ,  12  G  I*, 

Li  1,d  1 2  implies  [runs(ti)J Ad  C  Lruns(<2)jAd 

Discussion.  Although  the  at-output-checking  formulation  allows  us  to  formalize  the  email  server 
policy,  at-input-checking  does  have  some  advantages.  Both  use  the  input  sequence  on  the  left- 
hand  side  to  produce  a  static  policy.  Given  this  sequence,  the  at-output-checking  formulation 
selects  one  such  static  policy  using  the  whole  input  sequence.  The  at-input-checking  formulation, 
however,  selects  a  new  static  policy  with  each  recursive  application.  This  allows  the  at-input- 
checking  formulation  more  flexibility  to  treat  each  input  of  the  sequence  differently  even  if  the 
inputs  come  from  the  same  domain. 
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A  related  limitation  of  at-output-checking  is  its  inability  to  capture  revocation,  the  removal  of 
a  previously  held  access  right.  For  example,  revocation  takes  place  if  df  dt  but  df  dt 

Under  the  at-input-checking  formulation,  this  would  mean  that  df  may  access  the  input  i\  but  not 
the  input  i 2.  However,  for  a  system  to  obey  the  policy  under  the  at-output-checking  formulation, 
the  system  must  not  produce  output  influenced  by  i\  for  df  even  if  the  output  is  produced  before 
12  arrives.  If  the  system  did,  it  would  lead  to  a  violation  of  the  policy  once  v2  arrives.  Thus,  for 
a  system  to  obey  the  above  policy,  it  must  actually  also  obey  the  policy  where  df  7AM  dt  and 
df  96 1*1  >*2]  dt.  For  this  reason,  at-output-checking  policies  cannot  express  revocation. 

We  defined  both  of  the  above  dynamic  formulations  to  depend  on  input  sequences  and  domains 
but  not  the  states  of  the  automaton,  making  them  input-based.  We  view  the  states  of  an  automaton 
to  be  implementation  specific  unlike  the  input-output  behavior  and  domains  of  the  system,  which 
are  at  the  specification  level.  Since  policies  should  be  at  the  specification  level,  we  avoided  referring 
to  the  states  in  the  definition  of  a  policy. 

Henceforth,  unless  otherwise  noted,  all  dynamic  policies  will  be  at-output-checking. 

4.3  Dynamic  Unwinding 

Unlike  policies  that  should  be  defined  without  reference  to  the  states  of  an  automaton,  unwinding 
conditions  must  be.  Thus,  we  need  a  version  of  dynamic  policy  that  depends  on  the  states  instead 
of  being  input-based.  Let  a  generic  state-based  dynamic  policy  ^  be  a  function  from  a  set  of  states 
to  a  relation  on  domains. 

To  give  the  unwinding  conditions  meaning  with  respect  to  an  input-based  policy,  we  must  relate 
the  input-based  policy  to  a  state-based  policy.  Let  the  state-based  dynamic  policy  be  a  safe 
approximation  of  a  input-based  dynamic  policy  iff  df  -/d  dt,  [a\i  =  1,  and  qo  q  implies 

df  /U  df.  We  call  non-revoking  iff  for  all  a  £  A*,  q-^+q'  and  df  q  dt  implies  that  df  ~-~>q  dt. 

Given  a  system  automaton,  let  a  dynamic  view  partition  be  a  function  from  a  pair  of  domains 
to  an  equivalence  relation  on  states.  That  is,  a  view  partition  is  in  D  x  D  — >  2^x<^.  We  will  write 

q\  ~  (/2  if  for  the  pair  of  domains  (dt,df),  the  states  q\  and  (72  are  within  the  relation.  Intuitively, 

df 

q\  ~  q-2  means  that  the  states  q\  and  q-2  should  look  the  same  to  dt  since  they  only  differ  by  secret 
df 

inputs  from  df. 

Let  a  dynamic  view  partitioning  •  ~  •  for  a  program  automaton  m  be  called  a  dynamic  unwind¬ 
ing  relation  for  a  state-based  dynamic  policy  if  ~  satisfies  the  following  dynamic  unwinding 
conditions: 

1.  Local  Respect:  for  all  dt ,  df  G  D,  q,q[  G  Q,  and  ?'iU2  G  I  if  dom(ii)  =  dom(*2)  =  df,  q-^->q[, 

dt 

and  df  dt,  then  there  must  exist  q2  £  Q  such  that  q  q'2  and  q[  ~  q2. 

dt  df 

2.  Step  Consistency:  for  all  dt,df  G  D,  q\ ,  q\ ,  q2  G  Q,  and  i  G  I,  if  qi~q2,  qi—^q[,  and 

df  dt 

df  /Ui  dt,  then  there  must  exist  q2  G  Q  such  that  q2  — q2  and  q[  ~  q2. 

dt  df 

3.  Output  Consistency:  for  all  dt,df  G  D,  qi,q[,q2  G  Q,  and  o  G  O  if  dom(o)  =  dt,  q\  ~q2,  and 

df 

qi  q\ ,  and  df  /Ui  dt,  then  there  must  exist  q2  G  Q  such  that  (72  — >  q'2  and  q[  ~  q2. 

dt  dt  df 
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As  with  static  unwinding  relations,  the  existence  of  a  dynamic  unwinding  relation  implies  that 
the  system  obeys  the  policy: 

Theorem  3.  For  all  automata  m,  if  is  a  non-revoking  safe  approximation  of  the  at- output¬ 
checking  DIINI  policy  and  there  exists  an  unwinding  relation  for  and  m,  then  m  obeys 

Appendix  B  provides  the  proof. 


5  Automated  Policy  Extraction 

Although  using  the  dynamic  unwinding  conditions  eases  proving  that  a  program  obeys  an  DIINI 
policy,  we  really  desire  an  automatic  algorithm  to  check  for  obedience.  Furthermore,  as  motived  in 
the  introduction,  often  one  would  like  to  know  the  most  restrictive  policy  that  a  program  obeys. 
Thus,  we  describe  an  approach  for  extracting  from  the  source  code  of  a  program  an  approximation 
of  the  most  restrictive  policy  obeyed  by  that  program. 

Our  approach  tracks  the  flow  of  information  through  the  program  in  a  manner  similar  to 
information-flow  type  systems  [29,  25].  However,  since  our  approach  must  work  for  legacy  code 
designed  without  the  analysis  in  mind,  some  of  the  limitations  of  these  type  systems  render  them 
unacceptable.  For  example,  type  systems  will  consider  high-level  any  information  stored  in  a  vari¬ 
able  that  has  ever  stored  high-level  information  even  if  the  current  information  stored  in  the  variable 
is  low-level.  Furthermore,  type  systems  make  no  attempt  to  rule  out  infeasible  paths. 

Thus,  we  approach  the  problem  with  model  checking.  For  each  ordered  pair  of  domains  df 
and  dt,  we  will  check  for  the  property  that  the  static  incident-insensitive  noninterference  assertion 
df  A  dt  is  not  violated  by  the  program.  The  collection  of  all  counterexamples  to  this  property  will 
form  all  the  executions  in  which  dt  gains  access  to  information  about  df.  From  these,  we  construct 
a  DIINI  policy  that  the  program  obeys. 

Our  approach  differs  from  standard  model  checking  in  that  we  need  all  of  the  counterexamples 
to  the  noninterference  property,  not  just  one.  Furthermore,  our  approach  differs  in  that  the  non¬ 
interference  property  is  neither  a  safety  nor  liveness  property  and,  thus,  not  expressible  in  any  of 
the  standard  temporal  logics  used  as  property  languages  [23].  Like  a  safety  property,  noninterfer¬ 
ence  requires  that  something  does  not  happen:  noninterference  is  not  violated.  However,  unlike  a 
safety  property,  to  determine  if  noninterference  is  violated  requires  comparing  two  behaviors  of  the 
program.  Thus,  Terauchi  and  Aiken  calls  noninterference  a  2-safety  property  [33]. 

To  address  the  first  difference,  we  use  an  all-counterexample  extension  to  standard  model  check¬ 
ing  [15,  30].  To  address  the  second  difference,  our  approach  constructs  a  model  of  the  program 
that  reifies  this  2-safety  property  as  a  normal  safety  property.  Before  presenting  this  construction 
formally,  we  provide  an  example.  In  the  example,  and  most  of  the  rest  of  the  section,  we  will 
only  concern  ourselves  with  extracting  the  dynamic  conditions  under  which  one  given  domain  gains 
access  to  one  other  given  domain.  We  discuss  extending  this  approach  to  more  than  two  domains 
in  Section  5.4. 

5.1  An  Example 

Consider  the  program  from  the  email  server  example  in  Section  4.1.  We  would  like  to  extract 
from  this  program  the  most  restrictive  DIINI  policy  that  it  obeys.  For  simplicity,  we  restrict  our 
attention  to  only  cases  where  the  user  (domain  u)  gains  access  to  the  emails  (domain  e).  Thus,  we 
will  model  check  for  the  property  that  the  static  policy  e  A  u  is  obeyed. 
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The  first  step  of  our  approach  performs  a  property-reifying  transformation  to  the  program 
making  the  2-safety  property  that  e  u  is  obeyed  into  a  safety  property.  For  each  variable  x,  the 
transformation  creates  a  shadow  variable  x1  that  tracks  if  x  is  independent  of  the  value  of  all  the 
inputs  produced  by  e.  The  transformed  program  is 

emails  =  load("mbox") ; 
emails’  =  false; 
real_pw  =  load("password") ; 
real_pw’  =  true; 
given_pw  =  read() ; 
given_pw’  =  true; 
if (given_pw  ==  real_pw) 
print (emails) ; 

print ’ (emails ’  &  given_pw’  &  real_pw’); 
else 

print ("wrong") ; 

print ’ (given_pw’  &  real_pw’); 

where  &  is  boolean  AND.  The  variable  emails’  is  the  shadow  variable  for  emails.  It  is  set  to 
false  because  emails  depends  on  an  input  from  e.  real_pw’,  the  shadow  variable  of  real_pw,  is 
set  to  true  since  it  is  independent  of  e.  Likewise  with  given_pw  ’ . 

print  ’  is  a  special  function  that  shadows  calls  to  print.  It  allows  us  to  reify  that  u  has  gained 
access  to  the  inputs  of  e  since  whenever  u  does,  print’  is  called  with  the  value  of  false. 

In  the  then  branch  of  the  if  statement,  print  ’  is  passed  emails  ’  &  given_pw’  &  real_pw’. 
It  is  passed  emails  since  the  print  statement  it  is  shadowing,  which  precedes  it,  directly  depends 
on  the  value  of  emails.  It  is  passed  given_pw’  and  real_pw’  since  by  being  in  an  if  statement 
whose  predicate  depends  on  these  values,  the  print  statement  indirectly  depends  them.  These 
three  shadow  variables  are  conjoined  since  all  three  of  them  must  be  independent  of  e  for  the  print 
statement  to  be  independent  of  e. 

The  print’  statement  in  the  else  branch  only  has  given_pw’  &  real_pw’  since  the  print 
statement  it  is  shadowing  only  depends  (indirectly)  on  these  values. 

Checking  for  the  safety  property  that  “print’  is  never  passed  the  value  of  false”  yields  a 
counterexample  whenever  given_pw  ==  real_pw.  This  condition  is  only  satisfied  when  the  contents 
of  password  is  equal  to  the  user’s  input.  Thus,  u  only  gains  access  to  the  input  of  e  if  the  input  of 
password  equals  the  input  of  u.  Therefore,  the  program  obeys  the  policy  where  e  d  u  when  the 
input  sequence  i  has  the  same  second  (real_pw)  and  third  input  (given_pw)  and  e  -/»'■  u  otherwise. 

We  can  use  the  same  method  to  extract  the  policy  that  governs  access  by  u  to  the  inputs  of 
password  (the  domain  p)  by  tracking  how  the  value  of  the  file  password  flows  through  the  system 
instead  of  how  the  value  of  mbox  does.  One  may  see  from  the  above  transformed  program,  that 
both  print  statements  depend  on  the  value  of  password.  Thus,  the  user  always  gets  access  to  the 
input  of  p.  Indeed,  the  user  does  learn  if  the  password  he  has  supplied  as  input  is  equal  to  value 
of  password  or  not.  In  practice,  this  small  bit  of  information  is  often  negligible,  a  concept  others 
have  formalized  [13,  18,  26],  but  we  consider  outside  the  scope  of  this  paper. 

Since  our  approach  relies  on  the  semantics  of  the  analyzed  language,  we  first  present  a  simple 
language  before  formalizing  our  approach  for  that  language. 
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{T,x:=e)  ^(T[x  T(e)],*) 

n  =  T(e) 

,  ,  (o ,d,n)  , 

(r,  print  (e,  d ))  <— >  (r,  •) 


(i  ,d,n) 

(r,read(x,  d ))  ^  (r[i'nn],«) 


(r,gl)^r,4) 

(r  ,s1-,s2)^(r',s'1;s2) 


(T,s2)^(T\s'2) 

(r,.;S2)A(r',.;4) 


r(e)  =  0  r(e)  +  0 

(r,  • ;  •)  ^>(T,  •)  (r,  if  (e)  {si}{s2})  ^(r,  s2)  (r,  if  (e)  {-Si}{s2})  ^(r,  s2) 


(r,  while  (e)  {si})  ^->(T,  if  (e)  {si ;  while  (e)  {si}}else{»}) 


Table  1 :  Semantics  of  WhilelO 

5.2  The  Language  WhilelO 

WhilelO  is  simple  language  with  while  loops,  if  statements,  and  operators  for  input  and  output. 
The  syntax  of  WhilelO  consists  of  statements  S  and  expressions  E: 

S  ::=  X:=E  |  print (E,  D)  |  read(X,  D)  |  S ; S 
|  if (E) {S}else{S}  |  while(E){S} 

E  ::=  E+E  j  X  |  D  |  N 

where  X  ranges  over  variable  names,  D  over  domains,  and  N  over  numbers.  Statements  always 
evaluate  to  void  (written  as  •),  and  expressions  always  evaluate  to  a  number.  A  program  is  just  a 
single  statement. 

Table  1  gives  the  semantics  of  WhilelO.  The  judgment  (T,  s )  ^->(T7,  s')  means  that  the  statement 
s  goes  to  s'  while  performing  the  action  a  and  changing  the  store  from  T  to  r7.  The  store  is  a 
mapping  from  variables  to  numbers:  T  :  X  —>  N.  Let  T[x  i— >  u]  be  the  store  such  that  T[x  t— >  v\(y ) 
is  u  if  x  =  y  and  is  T(y)  if  x  ^  y.  We  extend  stores  to  assign  a  number  to  expressions  as  follows: 
let  T(ei+e2)  be  T(ei)  +  T(e2)  and  T(n)  =  n  for  numbers  n. 

An  action  is  an  ordered  triple:  the  first  component  is  i  if  the  action  is  an  input  and  o  if  it  is  an 
output,  the  second  component  is  the  domain  of  the  action,  and  the  third  component  is  the  contents 
of  the  action.  For  example,  (i,e,  "Dear  Bob.  .  .  ")  could  be  the  input  for  the  emails  in  the  email 
server  above  example. 

A  program  of  WhilelO  defines  an  automaton.  The  inputs  I  are  those  actions  with  i  as  the  first 
component;  the  outputs  O,  those  with  o  as  the  first  component,  dom  projects  the  second  component 
of  an  action.  Each  pair  (T,  s)  defines  a  state.  The  transitions  are  provided  by  the  judgment  form 
c— k  (T,  s)  ->(F',  s')  iff  (T,  s)  >(T7,  s7).  The  initial  state  is  (To,s)  where  s  is  the  program  and  To 
is  the  store  that  assigns  zero  to  every  variable.  Given  a  program  s  let  autom(s)  represent  this 
automaton. 

A  program  s  obeys  a  DIINI  policy  iff  autom(s)  obeys  the  DIINI  policy  as  defined  in  Section  4.2. 
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5.3  Constructing  the  Model 

Now  we  show  how  to  convert  a  program  of  WhilelO  to  an  automaton  model.  Rather  than  preform 
a  source-to-source  transformation  as  in  the  example  of  Section  5.1,  we  show  how  to  reify  the 
noninterference  property  directly  in  the  model.  Thus,  the  model  contains  some  features  that  are 
unnecessary  for  simply  modeling  the  behavior  of  the  program.  Strictly  speaking,  these  extra  features 
mean  that  the  model  is  not  a  system  automaton  as  defined  in  Section  2. 

We  present  the  model  construction  algorithm  for  finding  the  conditions  under  which  the  confi¬ 
dentiality  assertion  df  -/»  dt  is  violated  for  a  fixed  pair  of  domains  (if  and  dt  such  that  dt  ^  df.  In 
the  next  section,  we  discuss  dealing  with  more  than  two  domains. 

Let  model(s)  =  (I,  O,  D,  dom,  Q,  (/o,  >—>)  be  the  model  constructed  for  the  program  s.  /,  O, 
D ,  and  dom  come  from  the  definition  of  action  found  in  Section  5.2.  The  set  of  states  Q  is 
( X  —>  N )  x  Ls  x  (I  {T,  F})  where  Ls  is  a  set  of  labels  defined  below.  Each  state  (T,  77)  6  Q 

consists  of  a  store  T,  a  label  £,  and  an  independence  predicate  rj. 

The  set  of  labels  Ls  for  atomic  statements  s  holds  just  two  labels:  pre(.s)  and  post(s),  which 
represent  the  state  right  before  executing  s  and  the  state  right  after.  The  set  of  labels  for  a 
compound  statement  s  (an  if,  while,  or  ;  statement)  results  from  adding  pre(s)  and  post(s)  to 
the  disjoint  union  of  the  sets  of  labels  for  its  sub-statements. 

At  a  state  (T,  £,  rj),  the  independence  predicate  ?/,  assigns  to  each  variable  x  true  if  at  that  state 
the  value  of  x  is  independent  of  the  value  of  any  input  from  the  domain  df.  If  x  does  depend  on 
the  value  of  an  input  from  df  or  it  is  unclear  if  it  does  or  not,  then  r/(x)  =  F.  Let  r/(e i+e-2)  be 
r](e  1)  A  r/(e 2)  and  77(77.)  =  F  for  n  €  N. 

The  start  state  qo  is  (To,  pre(s),  i]j)  where  s  is  the  program  and  r/j  is  the  independence  predicate 
that  assigns  true  to  all  variables. 

a 

>— »  is  a  transition  relation  from  a  state  to  a  state  under  both  an  action  and  a  boolean.  7  q 
means  that  the  model  transitions  from  state  q  to  state  q'  during  the  action  a  without  providing  any 

a 

information  about  df  to  dt  •  q  ^  q  means  that  the  model  transitions  from  q  to  q  during  a  while 
possibly  providing  information  about  df  to  dt. 

To  define  >— >,  we  use  a  translation  from  a  statement  to  a  transition  relation.  We  write  >s> 

a 

for  the  translation  of  s.  We  write  q>s>q  if  the  state  q  transitions  to  q  under  the  action  a  and 

b 

boolean  b  in  the  transition  relation  >s>.  The  value  of  >— »  for  the  program  s  is  >s>. 

The  translation  >s>  is  defined  recursively  on  the  structure  of  s.  For  each  syntactic  form  that  a 

a 

statement  can  take,  we  provide  all  the  cases  in  which  >s>  holds:  if  q  >s>  q  is  not  explicitly  listed, 

b 

then  it  does  not  hold  (is  not  in  the  relation).  (All  variables  are  universally  quantified.) 

1.  When  s  has  the  form  x:=e: 


T 

(T,  pre(s),  77)  >s>(T[x  T(e)],  post(s),  r)[x  i-»  77(e)]) 

2.  When  s  has  the  form  read(x,  d )  with  dt  /  d  /  df: 

(i  ,d,n) 

(T,  pre(.s),  77)  >s>(T[x  n],  post(s),  q[x  T]) 
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3.  When  s  has  the  form  read(x,  d )  with  d  =  df  or  d  =  dt: 


(i  ,d,n) 

(r,  pre(s),  rj)  >s>(r[x  i->-  n],  post (s),rj[x  i->  F]) 


4. 


When  s  has  the  form  print  (e,  d )  with  d  ^  dt: 

<o,d,r(e)> 

(r ,  pre(s),  77)  >s> 


(r,post(s),7?) 


5.  When  s  has  the  form  print (e,  dt): 


(r,pre(s),7/) 


(o,dt,r(e)) 

>s>  (r ,  post(s),  77) 

77(e) 


6.  When  s  has  the  form  *•]  ;  s2: 


(r,  pre(.s),  rj)  >s>{T,  pre(si),  77) 

(r,  post(si),  77)  >s>(r,  pre(s2),  v) 

(r,  post (s2),ri)  >s>{T ,  post(s),  77) 

a  .  a  .  a  . 

q>s>q  if  q>Si>q  or  q>s2>q 
b  b  b 

7.  When  s  has  the  form  if  (e)  si  else  s2: 

(r,  pre(s),  77)  >s>  (r,  pre(sj),  7/} 

r)(e)\/w 

(r,  post (Sj),  77)  >s>(r,  post(s),  rf) 

a  .  a  . 

q>s>q  if  q>Sj>q 
b  b 

where  j  =  1  if  T(e)  /  0  and  j  =  2  if  T(e)  =  0,  and  rj'{x)  =  77(2;)  A  (77(e)  V a;  ^  def(si)  U  def(s2)) 
where  def(s)  is  the  set  containing  all  variables  defined  (on  the  left-hand  side  of  a  :  =  statement 
or  the  variable  in  a  read  statement)  in  s,  and  w  is  false  if  si  or  s2  contain  a  while  loop,  a 
read  statement,  or  a  statement  of  the  form  print (e,  (it)- 

8.  When  s  has  the  form  while (e)si  with  T(e)  7^  0: 

(r,  pre(s),  77)  >s>(r.  pre(si),  77) 

77(e) 

(r.  post(si),  77)  >s>(r,  pre(s),  rj) 

77(e) 

a  j  a  . 

q>s>q  if  q>s\>q 
b  b 
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9.  When  s  has  the  form  while  (e)-Si  with  T(e)  =  0: 

(r,  pre(s),  rj)  >s>(T ,  post (s),rj) 

V{e) 

The  transitions  for  while  statements  produce  the  boolean  rj(e)  despite  producing  no  output  since 
their  termination  or  lack  there  of  may  affect  the  output  seen  by  the  user,  while  and  read  statements 
are  treated  specially  in  if  statements  for  the  same  reason. 

5.4  Using  the  Model 

Once  model  (s)  has  been  constructed,  our  approach  uses  it  to  create  an  approximation  of  the  most 
restrictive  DIINI  policy  that  the  program  s  obeys.  First,  our  approach  finds  all  reachable  transitions 

O 

of  the  form  q\  >— ►  q2  .  These  transitions  indicate  that  the  output  o  might  provide  the  low-level  user 
dt  with  information  about  an  input  of  df.  Second,  for  each  such  transition,  our  approach  finds 
each  input  sequence  i  that  leads  to  this  transition.  Third,  for  each  such  i,  df  dt  is  added  to 
the  policy  for  every  i'  that  has  t  as  a  prefix.  After  this  process  is  complete,  the  resulting  policy  is 
returned  with  df  -/>'  dt  for  all  t  such  that  df  dt  was  not  added  to  the  policy.  Let  policy(model(s)) 
represent  this  policy.  (We  define  policy(model(s))  more  formally  in  Appendix  C.) 

Correctness  of  our  approach  may  be  stated  as  follows: 

Theorem  4.  For  every  program  s  of  WhilelO,  autom(s)  obeys  the  DIINI  policy  policy(model(s)). 

We  prove  this  theorem  in  Appendix  C. 

To  convert  our  approach  to  an  actual  algorithm,  we  must  select  a  method  for  finding  all  the 

O 

input  sequences  that  lead  to  a  transition  of  the  form  giy®-  Finding  these  sequences  is  equivalent 

to  finding  all  the  counterexamples  to  the  property  no  such  transition  is  reachable.  While  standard 
model  checkers  will  stop  after  finding  one  counterexample  to  this  property,  algorithms  exist  for 
producing  all  the  counterexamples.  Jha  and  Wing  [15]  give  an  algorithm  using  a  symbolic  repre¬ 
sentation  of  the  state  space  and  a  modified  version  of  a  standard  iterative  fixed-point  algorithm  [24] . 
Sheyner  [30]  gives  another  algorithm  uses  an  explicit  state  representation. 

For  handling  more  than  the  two  domains  df  and  dt,  a  tool  can  repeat  the  above  approach  for 
each  ordered  pair  of  domains.  The  transitive  closure  of  the  union  of  these  policies  provides  a  policy 
that  the  program  obeys. 

6  Related  Work  and  Discussion 

Assumptions.  All  the  systems  discussed  in  this  paper  have  been  interactive,  that  is,  they  receive 
input  and  produce  output  throughout  their  execution.  A  batch-job  system  only  allows  users  to 
determine  the  contents  of  its  memory  at  the  beginning  of  its  execution  and  to  observe  any  changes 
at  the  end  of  its  execution.  Much  of  the  work  on  type  systems  for  enforcing  confidentiality  policies 
have  been  for  batch-job  systems  [29]. 

We  have  assumed  that  the  user  may  observe  not  only  the  outputs  from  a  system  but  also  the 
system  consuming  his  inputs.  Many  systems  actually  buffer  user  inputs  making  it  unclear  when  an 
input  actually  affects  the  state  of  the  system.  McCullough  discusses  some  issues  that  arise  from 
modeling  systems  that  use  buffers  [21]. 
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A  system  is  input-enabled  if  it  will  always  accept  any  input  offered  by  a  user.  While  most 
confidentiality  requirements  have  been  defined  for  input-enabled  systems,  we  have  not  made  this 
assumption. 

We  have  modeled  systems  as  asynchronous  automata,  which  can  provide  output  to  one  user 
without  sending  output  to  all  the  users.  Most  authors  use  synchronous  automata,  which  must 
produce  outputs  to  all  users  at  regular  intervals.  (See  [9]  for  a  detailed  comparison.)  We  believe 
our  unwinding  conditions  to  be  the  first  for  asynchronous  automata. 

We  have  assumed  that  the  users  cannot  observe  the  termination  of  a  system.  This  assumption 
makes  our  incident-insensitive  noninterference  requirement  termination-insensitive.  Others  have 
considered  program  analysis  for  termination-sensitive  confidentiality  requirements  [33]. 

Other  Requirements.  Incident-insensitive  noninterference  requires  that  if  i\  l/2,  then  any 
behavior  of  the  system  under  i\  must  also  appear  possible  under  ii  to  the  domain  d.  Thus,  this 
formulation  is  called  possibilistic.  In  some  contexts,  a  system  is  unacceptable  if  the  observations  of 
d  is  likely  to  occur  under  q  and  unlikely  under  l^.  Such  concerns  has  led  Gray  and  Syverson  to 
define  probabilistic  noninterference,  which  requires  the  observation  to  be  equiprobable  under  both 
i\  and  t,2  [14]. 

Nondeducibility  on  strategies  requires  that  no  matter  how  a  high-level  user  interacts  with  a 
system,  a  low-level  user  will  still  not  be  able  to  learn  anything  about  the  high-level  user’s  inputs  [36]. 
The  original  formulation  is  incident-sensitive.  O’Neill  et  al.  created  an  incident-insensitive  version 
to  characterize  formally  the  properties  that  information-flow  type  systems  enforce  for  interactive 
systems  [25].  We  suspect  that  few  if  any  modifications  would  be  required  to  use  our  approach  for 
extracting  nondeducibility  policies. 

Even  if  two  automata  obey  the  same  noninterference  policy,  their  composition  might  not.  Mc¬ 
Cullough  has  proposed  requirements  that  ensure  that  the  composition  of  two  obeying  automata 
will  also  obey  a  policy  [20,  21].  Also,  removing  nondeterminism  from  an  automaton  that  obeys 
a  noninterference  policy  might  result  in  one  that  does  not.  Others  have  studied  conditions  under 
which  such  refinement  will  not  destroy  the  security  of  an  automaton  [16,  19,  1]. 

We  have  required  that  each  policy  be  transitive.  Intransitive  policies  model  channel  control, 
the  requirement  that  information  passes  through  a  downgrading  domain  before  reaching  a  domain 
of  a  lower  level.  Rusby  defined  the  most  commonly  used  formulation  of  intransitive  noninterfer¬ 
ence  [28].  However,  Roscoe  and  Goldsmith  [27]  offer  a  competing  formulation  using  CSP  [11]. 

Whereas  confidentiality  requires  that  protected  data  does  not  become  known  to  untrusted  users, 
integrity  requires  that  protected  data  does  not  become  tainted  or  corrupted  by  untrusted  users.  By 
reversing  the  roles  of  the  high-  and  low-level  users  of  a  system,  integrity  becomes  confidentiality. 
Thus,  our  confidentiality  requirements  also  define  an  integrity  requirements. 

Dynamic  Unwinding.  Leslie  has  also  provided  a  set  of  dynamic  unwinding  conditions  [17]. 
Rather  than  asynchronous,  nondeterministic  automata,  she  defines  her  unwinding  conditions  for 
synchronous,  deterministic  automata.  Her  conditions  ensure  that  an  intransitive  incident-sensitive 
noninterference  policy  is  obeyed  while  ours  is  for  transitive  incident-insensitive  noninterference. 
Furthermore,  hers  is  for  at-input-checking  dynamic  policies  rather  than  at-output-checking  dynamic 
policies. 
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Related  Tools.  Although  we  are  the  first  to  propose  using  all-counterexamples  model  checking 
for  policy  extraction,  others  have  used  standard  model  checking  for  verifying  that  a  given  policy  is 
obeyed.  They  observed  that  by  composing  a  program  with  itself,  one  can  obtain  the  two  behaviors 
necessary  to  check  the  2-safety  property  of  noninterference  [3,  2] .  Later  work  improved  this  approach 
by  using  type  theory  to  produce  more  efficient  models  [33,  35]. 

Program  dependence  graphs  represent  how  inputs  from  different  users  interact  [4,  5].  Thus,  they 
reveal  if  a  system  obeys  a  noninterference  policy  [32] .  Hammer  et  al.  have  extended  this  approach 
also  to  produce  “witnesses”  (counterexamples)  in  cases  where  the  policy  fails  to  hold  [10].  These 
counterexamples  could  form  the  basis  of  an  algorithm  for  dynamic  policy  extraction. 

Just  as  a  confidentiality  policy  may  become  buried  within  the  code  of  a  large  program,  the 
operating  procedures  of  a  business  may  also  become  hidden  within  large  applications.  Thus,  others 
have  created  tools  to  extract  these  business  rules  from  source  code  [12,  31].  These  tools  use  program 
slicing  [34]  instead  of  model  checking. 

Once  a  policy  is  extracted  from  a  program,  the  maintainer  might  want  to  update  the  program 
to  accept  the  policy  as  a  configuration  parameter.  This  requires  refactoring  the  code  to  use  a 
centralized  policy  enforcement  mechanism.  Ganapathy  et  al.  have  developed  tools  to  retrofit  legacy 
code  for  this  purpose  [6]. 

7  Summary 

Firstly,  we  have  clarified  the  difference  between  incident-sensitive  and  incident-insensitive  noninter¬ 
ference,  two  requirements  often  conflated  as  simply  “noninterference” .  Secondly,  we  have  introduced 
at-output-checking  dynamic  policies  to  express  policies  that  at-input-checking  dynamic  policies  can¬ 
not.  Thirdly,  we  have  presented  an  approach  based  on  all-counterexamples  model  checking  for  the 
automated  extraction  of  at-output-checking  dynamic  incident-insensitive  noninterference  policies 
from  program  source  code. 
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A  Proofs  about  Static  Noninterference 

A.l  Formalization  of  Example 

We  formalize  the  example  found  in  Section  3.1.  We  model  this  program  as  the  system  exs  where 

•  I  exs  =  {T  5  F}j 

•  Oexs  =  {x,y,z}, 

•  Dexs  =  {H,  L}, 

•  domexs(T)  =  domexs(F)  =  H 
domexs(x)  =  domexs(y)  =  domexs(z)  =  L, 

•  Qexs  =  {qo,  91,92}, 

T  F  x 

•  and  the  transition  ->  is  such  that  qo^  9i,  9o^qi,  and  q\  ->  q-2 

exs 

where  q$  is  the  start  state.  The  system  only  accepts  input  from  the  domain  H  and  only  produces 
output  for  the  domain  L.  It  only  has  two  behaviors  [T,x]  and  [F,x].  Each  consumes  an  input  from 
the  domain  FI  and  then  produces  the  output  x  for  the  domain  L. 

The  desire  of  the  system  designer  is  to  protect  the  confidentiality  of  the  domain  H  from  the 
domain  L.  So  let  -^exp  be  a  policy  such  that  H  Aexp  L  and  L  ~~>exp  H.  This  policy  makes  H  a 
high-level  domain  and  L  a  low-level  domain. 

Lemma  1.  The  system  exs  fails  to  obey  ^>exp  as  incident- sensitive  noninterference  policy. 

Proof.  Both  []  and  [T]  are  in  I*  and  []  =f*p,L  [T]  since  domexs(T)  =  H  and  H  Aexp  L.  Thus,  it  should 
be  the  case  that  Lrunsexs([])Jj4L  =  [runsexs([T])Jj4L.  However,  [runsexs([])J al  =  L{}Jal  =  {}  whereas 
l_runsexs([T])JAL  =  L{[T,x]}Jal  =  {[x]}.  □ 
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Lemma  2.  The  system  exs  does  obey  exp  as  incident-insensitive  noninterference  policy. 

Proof.  For  H,  i.\  =n  exp’  t2  only  if  i\  =  c2  for  all  ti,  c2  G  I*  since  domexs(i)  d  for  all  d  G  Dexs  and 
i  G  /exs-  Thus,  clearly,  L\  ^*ex'”H  i2  implies  that  |_runseXs(/i)J  ahxs  =  Lrunsexs(i2)J ahxs- 
For  L,  consider  the  following  two  cases: 

1.  L\  G  {[T],  [F]}.  Then  i\  =^*exp’L  t2  if  anc[  only  if  i2  G  { [T] ,  [F] }  since  no  other  input  sequences 
of  length  one  exists  and  domexs(T)  =  domexs(F)  =  FI  and  H  7^exp  L.  Note 

Lrunsexs([T])JALxs  =  L{  [T,  x]  }J  ^ 

=  {L[T,x]jALj  =  {[x]}  =  {L[F,x]jALj 

=  L{[F,x]}Jalxs  =  Lrunsexs([F])J  alxs 

since  T  and  F  are  not  in  AgXS.  Thus,  |_r u nsexs (^-l ) J ^4^  =  |_runseXs(i2)J  alxs  if  <q  =^*exp,L  z,2. 

2.  L\  £  {[T],  [F] } .  Then,  as  explained  above,  <-2  ^  { [T] ,  [F] }  since  i\  =^exp,L  i2.  Thus, 

LrunSexs(ti)JALxs  =  L{}J  AeLxs  =  Lrunsexs(^2)jAeLxs 

since  no  behavior  of  exs  includes  neither  the  input  sequence  [T]  nor  the  input  sequence  [F] . 

□ 


A. 2  Proof  of  Theorem  1 

Lemma  3.  For  a  system  m,  for  all  d  G  D  and  ctq,  a2  £  I* , 

=n  0L2  implies  a\  =IS  ol2 

Proof.  Proof  by  induction  over  the  length  of  ctq.  Note  that  if  ctq  ctq,  then  |ctq|  =  |ck2|. 

Base  Case:  |ctq|  =0  and  ctq  =  [].  Then  ctq  must  be  [].  Thus,  ctq  ctq  since  []  =ff,d  []. 
Inductive  Case:  |ctq|  =  n  >  0.  Here  we  may  assume  that  ctq  =  a\\af  for  some  oq  G  A  and 
a\  G  T,  a2  =  a2:a2  f°r  some  a2  G  A  and  cc2  G  A*,  and  that  a\  =ff’a  a2  implies  that  of  =ff’a  a2. 
We  must  show  that  a\  :ol\  =^,(l  a2:a2  implies  that  a\  :a\  =ff’a  a2:a2 

Assume  ctq  =^’  or2 .  Then  dom(ai)  =  dom(a2),  dom(ai)  d  implies  that  a\  =  a2,  and 
of  =ff’d  a2.  Thus,  of  =ff,d  o!2.  Consider  the  following  two  cases 

1.  dom(ai)  7A  d.  In  this  case,  a2  =  a±.  Since  a\  =ff,d  o'2 ,  a\:af  =ff,d  a2:a2. 

2.  dom(ai)  7^  d.  In  this  case,  dom(a2)  7^  d  also  since  dom(ai)  =  dom(a2).  Thus,  a'x  =ff’d  a2 
implies  that  a\:af  a'2 ,  which  implies  that  af  =ff'a  cc2  implies  that  a\:of  =ff’a  a2:a2. 

Thus,  either  way,  ai'.a^  =ff,d  a2:a2.  □ 

Now  the  proof  of  Theorem  1. 

Proof.  Assume  that  a  system  m.  obeys  ^  as  a  noninterference  policy.  This  implies  that  if  ctq  =  ff’d 
02,  then  |/uns(ctq)J  Ad  =  Lruns(a2)J Ad-  By  Lemma  3,  if  a \  =ff’d  a2,  then  a \  =ff’d  a2.  Thus, 
if  ctq  =^’  ctq,  then  |_njns(ctq)J Ad  =  Lruns(a2)J Ad-  This  means  that  m  obeys  as  an  incident- 
insensitive  noninterference  policy. 

Lemmas  1  and  2  show  that  the  converse  is  not  true.  □ 
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A. 3  Proof  of  Theorem  2 

In  each  of  the  following  lemmas  let  •  ~  •  be  an  unwinding  relation  for  the  automaton 

m  =  (/,  O,  D ,  dom,  Q,  q0,  ->) 


and  policy 

Lemma  4  (Step  Respect).  For  all  d  G  D,  i\,i 2  G  I,  and  q\ .  q\ .  q2  G  Q,  i/dom(ii)  =  dom(*2), 
dom(ii)  7 U  d,  q\  ~  q2,  and  q\  -^0  q\ ,  then  there  must  exist  q'2  G  Q  such  that  q2  q2  and  q\  ~  q2  ■ 


Proof.  By  Step  Consistency,  there  must  exist  a  q2  such  that  q2  — A  q2  and  q\  ~  off.  By  Local  Respect 

n 


d 

d  ,  ,  ,  . .  .  . .  n  d 


there  must  exist  a  q'2  such  that  q2  — A  q2  and  q2  ~  q2  •  By  the  transitivity  of 

d 


/  d  1 

%  9l~^2- 


□ 


Each  of  the  next  five  lemmas  proves  almost  the  same  statement  for  a  more  complicated  set  of 
behaviors  than  the  last. 


Lemma  5.  For  all  d  G  D,  o  G  O,  a\  G  O* ,  and  q\ ,  q\ .  q2  G  Q,  if  dom  (o)  =  d,  L'RJa'*  =  []; 
qi^q-2,  and  q\  1  (f{ ;  i/jen  there  must  exist  a2  G  O*  and  q'2  £  Q  such  that  q2  ~^q2,  q[  ~q2,  and 

k2_Ud  =  D- 


Proof.  Since  = 

exist  q2  such  that  q2 
and  [<J2\Ad  =  []. 


:  [],  q\  — !->  q\  implies  that  q\  q[.  Thus,  by  Output  Consistency,  there  must 

d 

q2  and  q[  ~  q'2.  q2  — —>  q2  implies  that  there  exists  a2  such  that  q2  q2 
d  d 

□ 


Lemma  6.  For  all  d  G  D,  i\,i2  G  I,  o\  GO*,  and  q\ ,  q\ .  q2  G  Q,  if  \_ai\Ad  =  0>  Qi  ~  Q2,  Qi  <i'\ , 
dom(ii)  =  dom(*2),  and  dom(ii)  d  implies  i\  =  i2,  then  there  must  exist  a2  G  O*  and  q2  G  Q 

such  that  q2  q2 ,  q[  ~  q2 ,  and  [a2\  Ad  =  [] . 


Proof.  Since  L^i J  =  []>  Qi~^Qi  implies  that  q\  —^>q[.  Consider  the  following  two  cases: 

d 

•  dom(ii)  d.  In  this  case  i\  =  i2.  Thus,  by  Step  Consistency,  there  must  exist  q2  such  that 

i\  /  i  /  d  f 

Q2—^q2  and  q1~q2. 

d 

•  dom(ii)  -/>  d.  In  this  case,  Step  Respect  (Lemma  4)  implies  the  same  thing. 

In  either  case,  q2  — implies  that  there  exists  02  such  that  q2  -^->q2  and  [a2]Ad  =  [].  □ 

d 

Lemma  7.  For  all  d  G  D,  o  G  O,  <Ti,  G  O* ,  and  q\ ,  q\ .  q2  G  Q,  if  dom(o)  =  d,  q\  ~  q2,  q\  — >  q\ , 
and  ad  =  \ai\Ad,  then  there  must  exist  a2  G  O*  and  q'2  G  Q  such  that  q2-^->q2,  q\  ~ q2 ,  and 
[a2\Ad  =  ad- 
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Proof.  Proof  by  induction  over  the  structure  of  a d- 

Case:  (Jd  =  []•  The  result  follows  directly  from  Lemma  5. 

Case:  In  this  case,  o\  must  have  the  form  o'^.d \o'{  where  \a\ J  A,i  =  []  and  Ad  = 

a  • o  (j'  •~/ 

a'f.  Since  q\  there  must  exist  q2  such  that  q\  — 


Q\ 


Qv 


By  Lemma  5,  there  must  exist  a2  G  O*  and  q2  G  Q  such  that  q2  — ■>  q2,  q'\  ~  q2)  and  |_°2J  Ad  =  []• 
By  the  inductive  hypothesis,  there  must  exist  o2  G  O*  and  q'2  G  Q  such  that  q2  q2 ,  q'\  ~  q2,  and 

K  \a*  =  <- 

Let  a 2  =  a2:o':a2.  q2  q2  and  q2-^q2  implies  q2-^Q2-  L°2-Ud  =  0  and  W'll  Ad  =  ad 
implies  \cr2\Ad  =  \g2:o'-.(j2\ Ad  =  d:ad  =  Od- 


u 

d 

□ 


Lemma  8.  For  all  d  G  D,  h,i2  G  I,  o\  G  O* ,  and  q\ ,  q\ ,  q2  G  Q,  if  q\~q2,  qi-^q[,  dom(*i)  = 
dom(i2),  and  dom(ii)  ^  d  implies  i\  =  i2,  then  there  must  exist  a2  G  O*  and  q2  G  Q  such  that 

q2a^ql2,  q'i^q'2,  and  Ad  =  [°2-i2\ Ad- 

Proof.  In  the  case  where  =  []>  the  result  follows  directly  from  Lemma  6. 

Otherwise,  <ti  has  the  form  a'v\o\a'[  where  a[,a2  E  O* ,  o  &  O,  dom(o)  =  d,  and  [crf\Ad  =  []. 


Since  q\  a— >  q[ ,  there  must  exist  qf  G  Q  such  that  q\ 

By  Lemma  7,  there  must  exist  a2  G  O*  and  q2  G  Q  such  that  q2  q2-  q'{  ~  q2,  and  \a\  :o\  A,i  = 

[a'2:°\Ad.  Since  q'[  4  q2:q'(  - — >  q\ ,  and  \_<Ti\  Ad  =  [],  Lemma  6  implies  that  there  exists  a'[  G  O  and 

q2inQ  such  that  q2  »  q2,  q[  ~  q2,  and  \_<J2\ Ad  =  []. 

Let  a2  =  a2:o:a2.  Since  q2  A—,  q2  -^—>q2,  q2  FPFf  q'2  where  q[~q2-  From  \v'l\o\Ad  =  \_u'2.o\Ad, 
Yai\ Ad  =  []  =  \p2\ Ad,  and  the  fact  that  dom(ii)  =  d  implies  i\  =  i2  is  reflexive),  it  follows  that 
[a\\i\\Ad  =  [a2:i2\A2-  d 

Lemma  9.  For  all  d  G  D,  q\ .  q2.  q\  G  Q,  t\,i2  G  I*,  h,i2  G  F  and  a\  G  A*,  if  iy.i\  =^'d  i2-i2, 
L«iJ/  =  i\ ,  q\  ~ q2,  and  q\  T— >  q’x,  then  there  exists  a2  G  A*  and  q2  G  Q  such  that  q[  ~ q2,  q2 
[a2\i  =  l-2,  and  [a\'.ii\Ad  =  [ ot2'-i2\Ad . 


&1  '-0\  /»  & \  ‘^1 
— 


Qv 


Q2'.I2  l 

— *  Q2’ 


Proof.  Proof  by  induction  over  the  structure  of  L\ . 

Case:  L\  =  [].  Since  L\:ii  =)T’d  L2'-i2,  ^2  must  be  [],  dom(ii)  =  dom^),  and  dom(ii)  d  implies 
i\  =  i2 .  Since  [ciij/  =  [«i],  there  must  exist  a\  G  O*  such  that  a\  =  Thus,  Lemma  8  implies 

that  there  must  exist  02  G  O*  and  q2  G  Q  such  that  q2  q2,  Q\  and  \&i'-i\\Ad  =  [cr2:i2\Ad- 
Since  \02\1  =  [],  L a2'-i2\i  =  t2.i2.  Thus,  the  result  holds  with  a2  =  (72- 

Case:  i\  =  i\ .  Since  t\  =^f’d  l2,  there  must  exist  i2  G  I  and  i'2  G  I*  such  that  t2  =  *2:4, 
i.\  =ff’d  t2,  dom(i/1)  =  dom(i2),  and  dom(i,1)  d  implies  i\  =  i2.  Since  =  l\  =  i\  there 

must  exist  (j\  G  O*  and  a)  G  A*  such  that  aj  =  <j\  :i\ :«),  Lq/iJ/  =  b'\  ■  and  q\  — d  qf  d-4  q\ . 

Since  dom(i/1)  =  dom^),  dom(i,1)  ^  d  implies  i\  =  i2,  q\  ~  q2 ,  and  gi  — Ig'/,  Lemma  8  implies 
that  there  exists  ^  ^  Q  and  <^2  G  O*  such  that  q'{~q2,  q2  q2  and  Lcri:*/ijAd  = 


Since  i.\  :i\  =ff’d  L2:i2l  [p'i\i  =  L'\  ■  Qi^q'v  and  9i 


qL*1 


,  the  inductive  hypothesis  implies  that 

there  must  exist  a'2  G  A*  and  q2  G  Q  such  that  q[~q2,  q2  -^—*q2i  la2\l  =  4)  and  L^i :^i J Ad  = 
La2:*2j  Ad- 
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j  ,  7  /  C*  cr2-^2  //  i  //  Q:2•^2  /  Q!2:^2  / 

Let  C*2  =  (J2:'l’2:a2-  ^mce  #2 - *  ^2  anC^  #2  - >  ^2’  #2  - *  ?2‘ 

4)  La2jAd  =  ^2 •  Since  [cri^'J  Ad  =  Ad  and  |_ck^ 7iJ 

LcJ2:4:Q;2:*2j Ad-  That  is,  [Q;l:*lJJ4d  =  LQ2:*2j Ad- 


Since  l2  =  i'2.if2  aRd  La2JAd  = 

=  Ad>  |01:*l:Q:l:hjAd  = 

□ 


Theorem  2  is  a  corollary  of  the  next  lemma. 


Lemma  10.  For  all  d  E  D,  i\.  i2  El*,  a\  E  A* ,  and  q\  E  Q,  if  L\  =ff’d  i2,  qo~^qi,  and  \pi\l  = 
t\,  then  there  exists  a2  E  A*  and  q2  E  Q  such  that  go  -^>g2;  \0t2\1  =  1*2,  and  |_ oti\Ad  —  LQ2jAd- 

Proof.  Consider  the  case  where  a\  =  with  a\  E  A*,  i\  E  I,  and  a\,a[  E  O* , 

o  E  O,  dom(o)  =  d,  and  \a\  J  A,i  =  [].  Since  qo—^qi,  there  must  exist  q\ ,  q'{  E  Q  such  that 


al:*l  /  <Tl -o  // 
<70  — >  <?1  — >  <?1  ' 


■qi- 


Since  [aijj  =  l\ .  it  must  be  the  case  that  i\  =  l\  vi\  where  if  =  [a\  J j .  Furthermore,  since 
i\  i2.  =^’“  i2.  This  implies  that  i2  must  have  the  form  i'2:i2  where  dom(ii)  =  dom(i2) 

and  dom(ii)  d  implies  that  i\  =  i2.  Thus,  by  Lemma  9,  there  must  exist  a2  E  A*  and  q2  E  Q 


such  that  q[  ~  q2,  go  q2,  [pt'2\ /  =  l2,  and  \ot\ hi  J  A,i  =  [a'2:i2\Ad. 

Since  q[  ~  q2  and  q[  — ^g",  Lemma  7  implies  that  there  must  exist  er2  €  O*  and  q2  E  Q  such 
that  (f2aj^q2,  gi'~g2,  and  L^iJ^  =  L^J Ad- 

Let  a2  =  a2:i2:a2:o.  Since  g0  aA—t  q'2  and  q'2^>  q2,  qo~^q-2-  L“2j  1  =  [a2:i2:a2:o\j  =  [a2\j:i2  = 
i'2:i2  =  l2.  Since  |_ai:hJAd  =  LQ2:*2jAd>  L°iJ Ad  =  L^J Adi  aRd  L°iJ Ad  =  [];  La2_Ud  =  L of2:i2:a2:o\Ad  = 
|_C^2  :?-2  J  Ad- L°"2  J  Ad'.O  =  ]_CKi  nil  J  Ad  •  L^lJ  Ad =  L<al:*l:crl:o:°rlJ  Ad  =  L^iJa1*' 

In  cases  where  ai  is  not  of  the  form  a,1:ii:<7i:o:cr,1,  some  subset  of  the  above  arguments  are 
sufficient  to  achieve  the  same  result.  □ 


B  Proof  of  Theorem  3 


First,  we  must  define  some  new  notation. 

Let  gi  ~  q2  for  d':5  E  D*  iff  gi  ~  g 3  and  g3  ~  q2.  Let  g  ~  g  hold  for  all  d  and  g.  For  D'  C  D,  let 
d':S  d!  &  |i 

gi  ~  q2  iff  there  exists  <5  E  (D')*  such  that  gi  ~?2- 

For  D'  C  D,  let  D'  -f*q  d  mean  that  for  all  d!  E  D1 ,  d'  -f*q  d. 


Lemma  11.  For  all  D'  C  D,  a  E  A*,  and  q.  q'  E  Q,  if  the  the  state-based  dynamic  policy  is  a 
non-revoking  safe  approximation  of  the  dynamic  policy  g  q' ,  and  D'  -/*q  d,  then  D'  -f^q  d. 

Proof.  Consider  each  d!  E  D'  separately,  this  follows  from  the  contrapositive  of  the  fact  that  is 
non-revoking.  □ 


Lemma  12.  Let  be  a  state-based  safe  approximation  of  the  dynamic  policy  for  some  automa¬ 
ton  m  =  (I,  O,  D,  dom,  Q,  go,  ->).  For  all  d  E  D,  q\  E  Q,  l\,l2  E  I*,  ii,i2  E  I,  and  a  E  A*,  if 

i\:i\  1  1,d  i2:i2,  [ot\  j  =  t\,  and  go  — +gi,  then  dom(ii)  d  implies  i\  =  i2. 

Proof.  Since  ti:*i  1  1,d  t2'i2,  dom(ii)  =  dom(z2)  and  dom(ii)  d  implies  i\  =  i2.  Since 

is  a  safe  approximation  of  and  go  ^>gi,  dom(ii)  qP>Li:n  d  implies  dom(ii)  -f*qi  d.  Thus,  by 
taking  the  contrapositive,  dom(ii)  qi  d  implies  dom(ii)  ■^>br-n  d.  This  means  that  dom(*i)  d 
implies  i\  =  i2.  □ 
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In  each  of  the  following  lemmas  let  •  ~  •  be  an  dynamic  unwinding  relation  for  the  automaton 

m  =  (I,  O,  D,  dom,  Q,  e/o,  -►)  and  state-based  dynamic  policy  where  is  a  non-revoking  safe 
approximation  of  the  dynamic  policy 

The  next  two  lemmas  just  raise  up  the  second  two  unwinding  conditions  to  work  over  sets. 
Lemma  13  (Set  Step  Consisteny).  For  all  d  G  D,  D'  C  D,  i  G  I,  and  qi,q[,q2  G  Q,  if  q\  ~  e/2, 

qi  e/j,  and  D'  -f*q i  d,  then  there  must  exist  q'0  G  Q  such  that  q2  -d->  q'2  and  q[  ~  q'9. 
d  d  D' 

Proof.  We  will  actually  prove  the  following  slightly  stronger  statement:  For  all  d  G  D,  D'  C  D, 

5  G  ( D ')* ,  i  G  I,  and  q\ .  q[,  q 2  G  Q,  if  q\  ~  <72 5  qi  —>  q[,  and  D'  -/+qi  d,  then  there  must  exist  q'2  G  Q 

S  d 

such  that  q2  q2  and  q[  ~  q2 . 

d  S 

Proof  by  induction  over  the  structure  of  5. 

Case:  6  =  [].  In  this  case  q\  =  q-2-  Thus,  let  q2  =  q[.  Then  q[  ~  q2  by  definition. 

Case:  5  =  d':5'.  In  this  case,  q\  ~e/3  and  73  ~e/2  for  some  d!  and  73.  Since  d!  G  Df  d'  -f*qi  d. 

d'  S' 

Thus,  by  Step  Consistency,  there  must  exist  a  q3  such  that  73  -d->  q3  and  q[  ~  q3.  By  the  inductive 

d  d' 

hypothesis,  there  must  exist  q2  G  Q  such  that  72  — — *•  q'2  and  q'3^q2-  Thus,  q[  ~  q2.  □ 

d  S'  d'\S' 

Lemma  14  (Set  Output  Consisteny).  For  all  d  G  D,  D'  C  D,  o  G  O,  and  qi,q[,q2  G  Q,  if 

dom(o)  =  d,  q\  ~  q2,  qi  — +q[,  and  D'  d,  then  there  must  exist  q2  G  Q  such  that  q2  — ee/(,  and 
D'  d  d 

/  d  , 

qi  ~  q2  ■ 

1  D'  Z 

Proof.  We  will  actually  prove  the  following  slightly  stronger  statement:  For  all  d  G  D,  D'  C  D, 

5  G  ( D ')*,  o  G  O,  and  q\ .  q\ .  q-2  G  Q,  if  dom(o)  =  d,  q\  ~  e/2,  q\  -^->e/(,  and  for  all  D'  -f*q  1  d,  then 

S  d 

there  must  exist  q2  G  Q  such  that  e/2  q2  and  q[  ~  q2 . 

d  S 

Proof  by  induction  over  the  structure  of  <5. 

Case:  S  =  [].  In  this  case  q\  =  q2-  Thus,  let  q'2  =  q\ .  Then  q\  ~  q2  by  definition. 

Case:  5  =  d':5' .  In  this  case,  q\  ~e/3  and  e/3  ~  q2  for  some  d!  and  e/3.  Since  d!  G  D' ,  d!  d. 

d!  6' 

Thus,  by  Output  Consistency,  there  must  exist  a  q3  such  that  e/3  q3  and  q[  ~  q3.  By  the  inductive 

d  d' 

hypothesis,  there  must  exist  q2  G  Q  such  that  q2  q'2  and  q3^q2-  Thus,  q\  ~  q'2.  □ 

d  °  S'  d’:S' 


Now  we  raise  Step  Respect  to  work  over  sets. 

Lemma  15  (Set  Step  Respect).  For  all  dt,df  G  D,  D'  C  D,  i\,i2  G  I,  and  q\ ,  q\ ,  q2  G  Q,  if 

dom(ii)  =  dom(i2)  =  df,  q\  ~  q2,  q\  df  G  D' ,  and  D'  -/*qi  dt,  then  there  must  exist  q2  G  Q 

D'  dt 

such  that  q-2  —F,  e/o  and  q\  ~  e/o . 

dt  D<  z 
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Proof.  Since  q\  ~  q2,  Set  Step  Consistency  (Lemma  13)  implies  that  there  must  exist  a  q2  such  that 


92 


'  q2  and  q\  ~  q".  By  Local  Respect,  there  must  exist  a  q'2  such  that  q2  — ^  q2  and  q2  ~  q2. 


dt 


D' 


dt 

dt  /  /  dt 


df 


Since  q[  ~  q2,  there  must  exist  5  G  {D')*  such  that  q[  ~  q2.  Since  q"  ~(/9,  q[  ~  q2 ■  Thus,  since 
D'  S  df  S:df 

d',d<eD',  q\  &<*'.  □ 

The  next  five  lemmas  mirror  the  corresponding  five  lemmas  (Lemmas  5  to  9)  of  Section  A. 3 
very  closely. 

Lemma  16.  For  all  d  G  D,  D'  C  D,  o  G  O,  o\  G  O* ,  and  qi,  q[,  q2  G  Q,  if  dom(o)  =  d,  L^iJa^  =  []; 
9i  ~  92;  9i  — l-+q[>  and  D'  d,  then  there  must  exist  a2  G  O*  and  q'2  G  Q  such  that  q2  -Df  q2, 
d 


Qi  ~92,  and  [ a2\Ad  =  []. 


D' 


Proof.  Since  L^iJa^  =  D>  9i~^9i  implies  that  Thus,  by  Set  Output  Consistency 


(Lemma  14),  there  must  exist  q2  such  that  q2 — >  q'2  and  q\  ~  q'2.  q2 — >q2  implies  that  there 

d  D'  d~ 

exists  (J2  such  that  q2  q2  and  L°"2 J  Arf  =  []• 


□ 


Lemma  17.  For  all  d  G  D,  D'  C  D,  i\,i2  G  I,  o\  G  O* ,  and  q\ .  q\ ,  q2  G  Q,  a  G  A*,  if  L^ijAd  =  []; 
9i  ~  92?  9i  q'\,  dom(ii)  =  dom(i2),  dom(ii)  ^  D'  implies  i\  =  i2,  and  D'  q 1  d,  then  there  must 


D> 


exist  a2  G  O*  and  q2  G  Q  such  that  q2  q'2>  q'{  ~  92;  and  Lcr2 J  Ad  =  []• 


Proof.  L^i J  Ad  =  [],  9i  — >  q'\  implies  that  q\  —^q[.  Consider  the  following  two  cases: 

d 

•  dom(f1)  G  D' .  Since  D'  -/>qi  d,  dom(i1)  -f>qi  d.  Thus,  by  Set  Step  Respect  (Lemma  15,  there 

must  exist  q2  such  that  q2  q2  and  q\  ~  q2 . 

d  D' 

•  d  D' .  Set  Step  Consistency  (Lemma  13)  implies  the  same  thing  in  this  case. 

In  either  case,  q2  q'2  implies  that  there  exists  a2  such  that  q2  -^fq2  and  [a2\  Ad  =  []•  □ 

d 

Lemma  18.  For  all  d^D,D'  CD,  o^O,  cri,<Jd  G  O* ,  and  9i ,  q\ ,  q2  G  Q,  if  dom(o)  =  d, 
9i  ~  92;  9i  — >  9i ,  Od  =  L^iJa^  and  D'  -f->q'i  d,  then  there  must  exist  a2  G  O*  and  q2  G  Q  such  that 

92^92;  9i~92;  and  [ a2\Ad  =  ad. 

Proof.  Proof  by  induction  over  the  structure  of  ad. 

Case:  ad  =  [].  The  result  follows  directly  from  Lemma  16. 

Case:  ad  =  o':a’f.  In  this  case,  o\  must  have  the  form  a'^.o'-.a'f  where  L^i J  Ad  =  []  and  Ya\\  Ad  = 

a"d.  Since  q±  -Df  q\ ,  there  must  exist  q2  such  that  q\  q'f  q[.  By  Lemma  11,  q'{  — >  q'x  implies 
D'  ^>q”  d. 
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./  .0/ 
2* 


By  Lemma  16,  there  must  exist  af  G  O*  and  g2  G  Q  such  that  g2  — >  g2 ,  g"  ~  qf,  and  = 


D' 


[].  By  the  inductive  hypothesis,  there  must  exist  af  G  0*  and  q'2  G  Q  such  that  q2  -d— >  qf,  qf  ~  qf, 
and  |u"JAd  =  °d- 


D' 


_/  / 

T  j  /  /  //  a2  ' 

Let  (72  =  <r2:o  :a2 


(J//-6 

.2^.u2.  q-2  — ’  g2  and  Q2^Q.2  implies  q2^qf.  L^Ja^  =  0  and  ^ 

implies  =  Lcr2:°/:cr2 J  Ad  =  °':(Jd  =  ad-  □ 


Lemma  19.  For  all  d  G  D,  D'  C  D,  i\,i2  G  /,  ui  G  0*,  and  gi ,  g^ ,  g2  G  Q,  if  qi  ~g2?  <li  qf, 
dom(ii)  =  dom(*2),  D'  ~f*q i  d,  and  dom(?'i)  ^  D'  implies  i\  =  i2,  then  there  must  exist  a2  G  O*  and 
q'2  &  Q  such  that  q2  <T^~^  qf,  (lf  ~  d2 ;  and  =  Lcr2-*2J - 

Proof.  In  the  case  where  |_crijAd  =  []>  the  result  follows  directly  from  Lemma  17. 

Otherwise,  0\  has  the  form  af:o:af  where  af,af  G  O*,  o  G  O,  dom(o)  =  d,  and  [cd/J^  =  []. 

Since  q\  >  g^,  there  must  exist  g"  G  Q  such  that  q±  d— >  qf  — >  g^ .  By  Lemma  11,  qf  — >  qf  implies 

ir  d. 

By  Lemma  18,  there  must  exist  a2  G  O*  and  g2  G  Q  such  that  q2  -d->  q2.  qf  ~  qf.  and  [af  :o\  A,i  = 
|_(r2 :  o  J  A,i .  Since  qf  ~  qf,  qf  d— i ^  qf,  and  L^iJa^  =  []>  Lemma  17  implies  that  there  exists  af  G  O 
and  qf  G  Q  such  that  qf  — l  qf ,  qf  ~  g2,  and  [af\  A<i  =  []• 


Let  a2  =  af:o:af.  Since  q2  —d^ qf  -^—>qf,  q2-^—*qf  where  qf  ~  qf .  From  \crf'.o\Ad  =  [pf'-o\Ad, 

[af\Ad  =  []  =  [(if\Ad,  and  the  fact  that  dom(?'i)  =  d  implies  i\  =  i2  is  reflexive,  so  d  ^  IT), 

it  follows  that  \ai'.ii\Ad  =  Lcr2^2jA2-  □ 

Lemma  20.  For  all  d  G  D,  D'  C  D,  gi  G  Q,  ti,t2  G  /*,  *i,i2  G  /,  and  a\  G  A*,  ?/  D'  = 
{  d'  G  H  |  d'  -f*qi  d},  ti:*i  1  1,d  i2:i2,  |_cti J /  =  ti,  and  qo°^—fqi,  then  there  exists  a2  G  A*  and 
q2  €  Q  such  that  gi  ~g2,  qo°^>  <?2,  |_a2.|/  =  <-2,  and  Lai:*iJAd  =  La2d2_|Ad- 

Proof.  Proof  by  induction  over  the  structure  of  i\ . 

Case:  i\  =  [].  Since  i\:i\  =7T  1  1,d  i2-i2,  Lemma  12  yields  that  dom(ii)  ~^>qi  d  implies  i\  =  i2. 
Also,  i2  must  be  [].  Since  D '  =  { d'  G  D  \  d!  ~f^qi  d },  dom(ii)  ^  D'  implies  i\  =  i2.  Since 
LaiJj  =  [ii],  there  must  exist  <7\  G  O*  such  that  a\  =  a\:i\.  Thus,  Lemma  19  implies  that  there 
must  exist  er2  G  O*  and  qf  G  Q  such  that  qo<-^—>q2l  qi  ~Q2,  and  =  \&2'-i2\Ad ■  Since 

L<t2J i  =  [],  \_(J2vi2\  j  =  i2:i2.  Thus,  the  result  holds  with  a-2  =  a2. 

Case:  i.\  =  if: if.  Since  i\  =,7  1  1,d  t2,  there  must  exist  if  G  I  and  if  G  I*  such  that  t2  =  if:if, 
tf  1  1,d  if,  dom(d1)  =  dom(i2),  and  dom(d1)  d  implies  if  =  if.  Following  the  same  logic 

as  above,  this  allows  us  to  conclude  that  dom(d1)  ^  D'  implies  if  =  if. 

Since  \ai\i  =  i\  =  if  :if ,  there  must  exist  a\  G  O*  and  af  G  A*  such  that  a\  =  a\ :if:af, 

(7 1  :h  .  <7p.ii  .  ,  / 

LaiJj  =  d] ,  and  go  — ^  yi  — #  gi-  By  Lemma  11,  qf  — >  q\  implies  D  d. 
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Since  dom(i'1)  =  dom(i2),  dom(i/1)  d  implies  i\  =  i'2 ,  1o~1o>  and  qo q[,  Lemma  19 
implies  that  there  exists  q2  G  Q  and  a2  G  O*  such  that  q[  ~  q2 ,  qo  — >  q2  and  [ci  AJ  Ad  =  [&2'-i2]  Ad . 


Since  t[:ii  t2:i2 ,  =  4)  1i  ~  I2 ;  and  li  >ii,  the  inductive  hypothesis  implies  that 


,  ap* l 


D 


there  must  exist  a2  G  A*  and  q2  &  Q  such  that  q\  ~(/2>  q2  — >  q2,  \p-'2\l  =  d2,  and  |_ck^ :«i J = 
[a2:i2\Ad. 

Let  a2  =  a2:i2:a2.  Since  qo q2  and  q2-^q2,  qo  12 ■  Since  i2  =  i2:t2  and  = 

i2,  \ot2\Ad  =  L2-  Since  L^iAJa^  =  la2-i2\Ad  and  :ii J =  [a2:i2\Ad,  [cri-.i'^.a'^.ii] Ad  = 
[a2:i2:a'2:i2\Ad.  That  is,  \ai:i\\Ad  =  la2 -i2\Ad.  □ 

Theorem  3  is  a  corollary  of  the  next  lemma. 

Lemma  21.  For  all  d  G  D,  l\,l2  G  I*,  a\  G  A*,  and  q\  G  Q,  if  Li  =^T  1,d  i2,  qo  — Lc/i,  and  |_aij/  = 
Li,  then  there  exists  a2  G  A*  and  q2  G  Q  such  that  qo  ~^q2,  |_a2j /  =  £ 2 ,  and  \a\\Ad  =  [a2\Ad. 

Proof.  Consider  the  case  where  a\  =  with  a\  G  A*,  i\  G  I,  <7i ,  a\  G  O*,  o  G  O, 

dom(o)  =  d,  and  L^iJa^  =  []•  Since  qo—^qi,  there  must  exist  q\ ,  <:([  G  Q  such  that 


al:*l  /  <ti:o  //  CTi 

lo  — >  li  — >  li  — >  1i 


Since  [anj/  =  U  and  a \  =  a[:ii:ai:o:a[,  it  must  be  the  case  that  i\  =  i\  vi\.  Furthermore  since 
t\  '  ■  i2.  t2  must  have  the  form  i'2.i2  for  some  i2  G  I*  and  i2  G  I.  Thus,  by  Lemma  12,  this 

means  that  dom(?'i)  d  implies  i\  =  i2.  Let  D'  =  { d'  G  D  \  d'  -fF1'1  d}.  By  Lemma  11,  af[  — - *q\ 
implies  D'  7A9"  d,  and  q[  q'f  implies  D'  d. 


By  Lemma  20,  there  must  exist  a'2  G  A*  and  q2  G  Q  such  that  q[  ~  g2 1  lo  ►  q'2 ,  La2_U  =  4? 

and  ^ •  'i r J Ad  —  j_ci2.?'2jyi<i* 

Since  q[  ~  q2  and  q\  — ^  q" .  Lemma  18  implies  that  there  must  exist  <72  G  O*  and  q2  G  Q  such 


D 

that  12^12,  l"  ~12,  and  L^iJ 

Let  a2  =  a2:i2:a2:o.  Since  qo°^-*  q2  and  g2  ^■q2,  qo  12-  |_a2_|i  =  L«2:*2:cr2:oJ/  =  = 

f2.i2  =  i2.  Since  Lai:*iJAd  =  L^iJa^  =  L^Ja^,  and  L^Ua-*  =  01? 

L°i2  J  ^4<i  =  \oL2.i2'.<J2:o\Ad  =  LQ2:*2j  Ad  •  Ad  :o  =  Lal  :*lJ  Ad  ■  L^lJ  Ad  :o  =  LQ1  :*l:<:rl  :0:(JiJ  Ad  =  l_CKl  J  Ad 

In  cases  where  a\  is  not  of  the  form  a,1:ii:o‘i:o:cj,1,  some  subset  of  the  above  arguments  are 
sufficient  to  achieve  the  same  result.  □ 


C  The  Correctness  of  Our  Approach 

First,  we  must  relate  model(s)  and  autom(s).  Given  the  model  model(s)  =  (I,  O,  D,  dom,  Q,  qo,  ^ >), 
let  3(model(s))  be  the  system  automaton  (I,  O,  D ,  dom,  Q,  qo,  ->)  where  q\  ->  q2  iff  11  12  or  q\  q2. 

Lemma  22.  For  all  programs  s,  3(model(s))  and  autom(s)  have  the  same  set  of  behaviors. 
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This  lemma  means  that  if  we  can  prove  that  3(model(s))  obeys  some  policy,  then  we  know  that 
autom(s)  obeys  that  policy. 

Now  we  must  more  formally  define  our  approach  to  producing  a  policy  from  model  (s). 

a:a  rr 

Let  q  q  iff 

a  r 

•  there  exists  b  such  that  either  q  >—>  q  or  q  w  q  ,  and 

b  b 


//  a  / 
2  >— >  q  . 


where  q 

a:a  rr 

Let  q  >+++  q  iff 
b 


q'  only  if  q  =  q' . 

q' 


•  q  >— >  q"  or  q  >— >  q",  and 
b  b 


•  q"^q'- 


where  q  q'  only  if  q  =  q' . 
b 

Let  statePolicy(s)  be  the  state-based  dynamic  policy  where  df  dt  iff  there  exists  £ 


Ql 


Ot-2 


A*  and  a  €  A  such  that  qo  q'  q"  q.  This  means  that  df  ~^q  dt  for  any  state  q  such  that 

it  is  reachable  from  a  transition  that  produces  the  boolean  F  and  that  transition  is  reachable. 

policy(s)  is  statePolicy(s)  lifted  from  working  on  states  to  input  sequences.  Let  policy(s)  be  the 
input-based  dynamic  policy  ^  where  df  dt  iff  there  exists  a\,oi2  £  A*,  and  a  €  A,  and  q  €  Q 

<*i  /  /[“I////  Q2  i  , 

such  that  qo  >^>  q  ,  q  >-«->  q  ,  q  >->-»->  q,  and  l  = 

Lemma  23.  For  all  programs  s,  statePolicy(s)  is  a  non-revoking  safe  approximation  of  policy(s) 
for  3(model(s)). 


Proof.  Let  policy(s)  be  and  statePolicy(s)  be  df  dt  iff  there  exists  a  €  A*,  and  a  €  A,  and 

a:a\i.  T1 

q,  and  l  =  If  qo — >q  in  3(model(s)),  then  it 


a  [aj 

q  €  Q  such  that  qo  >-«->  q',  q'  q,  and  i  =  [a:a\i.  Thus,  if  df  dt,  then  there  does  not  a  €  A*, 


a  €  A,  and  q  G  Q  such  that  qo 


q' ;  q'  >->-»->  ( 


[aj 

T 


must  not  be  the  case  that  qo  q'  and  q'  > ■*>->  q.  Thus,  df  dt. 

It  is  non-revoking  because  of  how  any  states  reachable  from  a  state  q  where  df 
added  to  statePolicy(model(s))  also  has  df  ^  dt  added. 


dt  has  been 

□ 


Before  proving  Theorem  4,  we  must  prove  that  for  all  programs  s,  3(model(s))  obeys  the  DIINI 
policy  policy(s).  Since  the  above  lemma  tells  us  that  statePolicy(s)  is  a  non-revoking  safe  approxi¬ 
mation  of  policy(s)  for  3(model(s)),  we  may  use  the  unwinding  conditions  to  prove  this.  First,  we 
explain  the  unwinding  relation  we  will  demonstrate,  and  then  we  prove  that  it  indeed  satisfies  each 
of  the  unwinding  conditions. 

Recall  that  we  have  limited  our  construction  to  extracting  the  policy  for  when  df  flows  df.  Thus, 
statePolicy(s)  has  di  q  d2  for  all  q  when  df  /  df  or  d2  7^  dt.  Thus,  the  unwinding  conditions 
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places  no  requirements  on  such  d\  and  d‘2-  That  is,  q\  ~  q2  must  only  be  defined  for  the  case  where 

d-2 

d\  =  dt  and  d2  =  df  for  our  unwinding  condition.  Thus,  to  streamline  notation,  we  usually  drop 
the  domains  and  just  write  q±  ~  q2- 

Given  two  stores  Ti  and  T2,  let  Ti  =v  T  2  iff  for  all  x  G  X  such  that  rj(x)  =  T,  Ti(x)  =  T2(x). 
Let  the  dynamic  view  partition  ~  be  such  that  (Ti,^i,r/i)  ~  (T2,  £2,112)  iff  £\  =  £2,  f?i  =  ??2,  and 
Ti  =‘ni  T 2 •  We  will  show  that  ~  is  an  unwinding  relation. 

Lemma  24.  ~  has  dynamic  local  respect  for  3(model(s))  and  statePolicy(model(s)). 

Proof.  Since  3(model(s))  is  constructed  from  model(s),  the  only  transitions  in  3(model(s))  of  the 

form  q-+qi  come  from  a  transition  in  model  (s)  of  the  form  q>-^qi  for  b  =  T  or  b  =  F.  Since 

b 

the  transitions  of  model(s)  come  from  >s>,  we  may  examine  the  definition  of  >•>  to  find  when 

ii 

transitions  of  the  form  q^qi  are  possible.  These  are  only  possible  when  there  exists  a  statement 

b 

s'  that  is  a  sub-statement  of  s  (or  equal  to  s )  such  that  -s'  has  form  read(x,  d).  Furthermore,  the 
state  q  must  have  the  form  (T,  pre(.s/),r/) 

By  requiring  dom(?'i)  to  be  df,  we  further  limit  of  the  form  of  s'  to  read(x,  df)  and  the  form 
of  i\  to  (i,df,ni  for  some  n\.  Also  the  boolean  b  must  be  T.  This  implies  that  q\  has  the  form 

(T[x  1— ►  ni],  post(s/),  r][x  1— >•  F]).  Thus,  if  q-^q'  in  3(model(s)),  it  is  because  q^>qi  in  model(s) 

dt  T 

where  i\  and  q±  are  of  the  above  form. 

For  another  input  12  to  be  such  that  dom(i2)  =  df,  it  must  have  the  form  (i,df,n2)  for  some 
U2-  Let  q2  =  (r[x  t— >  712],  post(s),  r/[x  1— >  F]).  By  the  construction  of  >s>,  q>^>q2-  Thus,  q—^>q2  hr 
3(model(s)). 

Since  ?/[x  1— ►  F](x)  =  F,  and  T[x  1— >  m]  and  r[x  712]  agree  on  all  other  variables,  T[x  1— >  m]  =v 
T[x  1  7  77-2].  Thus,  q\  ~(/2-  □ 

To  prove  step  consistency  we  must  strengthen  the  hypothesis  and  introduce  some  additional 
concepts. 

<!  if  whenever  df  dt,  df  dt.  Note  that  may  be  defined  for  more  states  than 

<  ~~>2  implies  that  if  2  is  defined  at  q  and  df  ~f^>\  df,  then  df  df. 

Let  q\  — 7  q-2  iff 
d 

•  qi  =  q-2, 

•  Qi  q'  and  Q1  — >  <72 1  or 

d 

•  Qi  Q.'  and  q'  — >  (72  where  dom(o)  /  d. 

d 

First  we  prove  a  key  lemma  to  Step  Consistency  and  Output  Consistency.  One  can  views  step 
Consistency  as  requiring  two  states  that  are  related  by  ~  will  transition  to  other  related  states. 
Likewise,  Output  Consistency  requires  that  two  related  states  transition  to  two  other  related  states 
after  producing  an  output.  Under  this  view,  the  next  lemma  requires  that  two  related  states 
transition  to  two  other  related  states  upon  finishing  the  execution  of  a  statement. 

Lemma  25.  For  all  statements  s  and  s'  where  3(model(s))  =  (I,  O,  D,  dom,  Q,  qo,  ->)  and  s'  is  a 
sub-statement  of  s,  state-based  dynamic  policies  stores  Tj ,  independence  predicates  rj ,  q\,  q\ ,  q2  € 
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Q ,  if  <statePolicy(model(.s)),  gi~(/2,  q\ — >q[,  q[  =  (T^,  post(s/),  q'),  and  df  7 dt,  then  there 

dt 

must  exist  q2  G  Q  such  that  q2  — >  q2  and  q[  ~  q'2 . 

dt  df 

Proof.  Since  q\  ~  q2  we  know  that  q\  has  the  form  (Ti ,  £,  77}  and  q2  the  form  (T2,i,  rj)  where  Ti  =v  ^ . 
Proof  by  induction  over  the  derivation  of  >s>. 

Case:  s  has  the  form  x:=e.  Since  q\ — >q[,  £  must  be  pre(s).  Furthermore,  r)  must  be 

dt 

r  1  [a:  h- >  Ti(e)]  and  r /  must  be  q[x  *—>  77(e)]. 

Let  q!2  =  (^[x  1— >  r2(e)],  post(s),  r/[x  t— >  77(e)]).  q2^q'2  by  the  construction  of  model(s).  If 
77(e)  =  T,  then  Ti(e)  =  T2(e)  since  Ti  =n  T2-  If  77(e)  =  F,  then  r/[x  1— >•  77(e)]  (x)  =  F  Either  way, 

Ti[x  i->  Ti(e)]  =^v(e)]  r^x  r2(e)].  Thus,  q[  ~q'2. 

Case:  s  has  the  form  read(x,  d )  with  df  7^  d  7^  df.  In  this  case,  q\  cannot  make  a  transition 
without  consuming  input.  Thus,  this  case  is  trivially  satisfied. 

Case:  s  has  the  form  readier,  d)  with  d  =  df  or  d  =  dt.  Ditto. 

Case:  s  has  the  form  print  (e,  d)  with  d  7^  dt.  Since  q\  — >  q[ .  £  must  be  pre(s).  Furthermore, 

dt 

r;  =  Ti  and  if  =  77.  Let  q'2  =  (T2,  post(s),77).  q2  ~^q2  by  the  construction  of  model(s)  and  q[  ~  q2. 

Case:  s  has  the  form  print  (e ,  dt) .  In  this  case  q±  cannot  make  a  transition  without  producing 
output  to  dt.  Thus,  the  statement  is  trivially  satisfied. 

Case:  s  has  the  form  sa ;  Sb-  Since  Ls  is  the  disjoint  union  of  LSa,  LSbl  and  {pre(s),  post(s)}, 
one  of  the  following  cases  must  hold: 

•  qi  and  q[  are  both  in  Qa.  statePolicy(model(s))  <  statePolicy(model(sa))  since  model(s)  has  at 
least  as  many  transitions  under  a  true  boolean  as  model  (sa).  The  needed  result  follows  from 
the  inductive  hypothesis. 

•  q\  and  q[  are  both  in  Q b.  statePolicy(model(s))  <  statePolicy(model(sb))  since  model(s)  has  at 
least  as  many  transitions  under  a  true  boolean  as  model  (sb)-  The  needed  result  follows  from 
the  inductive  hypothesis. 

•  q\  G  Qa  and  q\  G  Qb-  Since  q\  — >  q2  and  the  construction  of  model (s),  there  must  exist  a 

d 

state  qia  of  the  form  (Tia,  post(sa),  ?7ia)  and  a  state  gib  of  the  form  (Tia,  pre(sb),  ?7ia )  such  that 
qi  — +  qu  ^qib— *q'v 

dt  dt 

By  the  inductive  hypothesis,  this  means  there  exists  a  state  q2a  such  that  q2  — 7  q2a  and 

dt 

(/ia  ~  q2a .  This  implies  that  the  form  of  q2a  is  (T2a,  post(sa), ?7ia) .  Thus,  by  the  construction 
of  model(s),  q2a^q2b  where  q2b  =  (T2a,  pre(sb),  7?ia)- 

Since  c/ib~g2b>  the  inductive  hypothesis  again  applies  and  there  must  exist  q2  such  that 

<?2b  — >  q2  and  q\  ~  q'2. 
dt 

•  </i  G  Qb  and  q[  G  Qa.  Since  q±  — >  q[  cannot  hold  in  this  case,  we  need  not  consider  it. 

dt 

•  q\  has  the  form  (Ti,  pre(s),  77)  and  q[  is  in  Qa  or  Qb.  Since  qi~q2,  q2  must  have  the  form 
(T  2 ,  pre(s) ,  77)  where  Ti  =r>  T2.  By  the  construction  of  3(model(s)),  qi-^-qia  where  gia  = 
(Ti,  pre(sa),77)  and  q2^q2a  where  q2a  =  (T2,  pre(sa),?7).  Since  q\a  ~g2a  and  they  are  both  in 
3(model(sa)),  the  proof  continues  as  above. 
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•  q\  is  in  Qa  or  Q b  and  q[  has  the  form  (T^,  post(s),  rf).  If  q\ — >  q[,  then  q\ — >gib  where 

dt  dt 

gib  =  (r/1,post(sb),r/').  Thus,  as  argued  above,  there  exists  g2b  =  (T^,  post(sb),  rf)  such  that 
gib~g2b-  By  the  construction  of  model(s),  gib-^gj  and  q2b^,Q2  where  q'2  =  (T^,  post(s),  rf). 
q[~q'2- 

•  gi  has  the  form  (Ti,  pr e(s),rj)  and  q[  has  the  form  (T7, ,  post(s),  rf).  In  this  case,  just  use  the 
arguments  found  in  the  two  cases  above. 

Case:  s  has  the  form  if  (e)saelsesb.  If  r/(e)  =  T,  then  Ti(e)  =  ^(e)  since  Id  =v  T2-  In  this 
case,  the  result  follows  from  using  the  inductive  hypothesis  on  sa  if  Ti(e)  /  0  and  on  sb  if  Ti(e)  =  0 
and  the  methods  used  above  for  dealing  with  the  cases  where  gi  has  the  form  (Id,  pr e(s),r/)  or  q[ 
has  the  form  (T^,  post(s),  ?/). 

The  same  holds  even  if  77(e)  =  F  as  long  and  Ti(e)  =  ^(e). 

If  df  ~^>qi  dt,  then  we  need  not  prove  anything  since  it  violates  a  premise  of  the  lemma.  Note 

that  if  r](e)  =  F  and  either  sa  or  sb  contained  a  while  loop,  a  read  statement,  or  a  statement  of 

the  form  print(e,  dt),  then  statePolicy(model(s))  would  allow  information  to  flow  from  df  to  dt  at 

q\.  Since  statePolicy(model(s))  is  non-revoking  and  gi  — >  q[,  the  same  would  be  true  at  q[.  Since 

d 

fdstatePolicy(model(s)),  df  dt  would  be  true.  Thus,  we  have  dealt  with  these  cases. 

This  leaves  the  case  where  rj(e)  =  F,  Ti(e)  /  ^(e),  and  neither  sa  nor  sb  contains  a  while  loop, 

a  read  statement,  or  a  statement  of  the  form  print  (e,  dt).  Since  there  are  no  read  statements, 

all  the  transitions  in  sa  and  sb  are  transitions  that  the  automaton  has  control  over  and  there  is  no 

chance  of  a  transition  being  blocked  by  a  user  not  offering  input.  Since  there  are  no  while  loops, 

once  sa  or  sb  is  entered,  they  will  surely  be  exited.  This  means  that  there  must  exist  q'2  such  that 

q2 — >q'2  and  q2  =  (T^,  post(s),  rj')  for  some  Tf,  and  rj'.  Since  Ti  =d  T2  and  r /  assigns  F  to  any 
dt 

variable  altered  in  either  sa  or  sb,  T7,  =r<  Y'2.  Thus,  q[  ~g^- 

Case:  s  has  the  form  while(e)sa  with  Ti(e)  7^  0.  If  ?/(e)  =  T,  then  Ti(e)  =  T2(e)  and  the 
inductive  hypothesis  may  be  applied  to  sa.  If  ?/(e)  =  F,  then  statePolicy(model(s))  would  allow 
information  to  flow  from  df  to  dt  at  gi .  As  above,  this  implies  that  df  dt  and  thus  the  result 
is  trivially  true. 

Case:  s  has  the  form  while  (e)sa  with  Ti(e)  =  0.  The  case  is  proved  as  the  previous  one  was. 

□ 


Now  we  prove  a  result  slightly  stronger  than  Step  Consistency. 

Lemma  26.  For  all  statements  s  where  3(model(s))  =  (I,  O,  D,  dom,  Q,  go,  ->•),  state-based  dynamic 

policies  gi ,  q\ ,  q2  G  Q,  and  i  G  I,  if  <statePolicy(model(s)),  gi  ~g2,  gi  — —>q[,  and  df  dt, 

'dt' 

then  there  must  exist  q2  G  Q  such  that  q2  q2  and  q[  ~  q2 . 

dt  df 

Proof.  Since  gi~g2  we  know  that  gi  has  the  form  (Ti,£,r])  and  (T 2 , 77)  where  Tf  =v  T2-  Let 

aV'-'/)- 

Proof  by  induction  over  the  derivation  of  >s>. 

Case:  s  has  the  form  x:=e.  In  this  case  gi  does  not  transition  to  any  other  state  under  an 
input  i  in  model  (s).  Thus,  the  statement  is  trivially  satisfied. 
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Case:  s  has  the  form  read(x,  d)  with  df  /  d  /  dt.  In  this  case,  for  <71  to  transition  to  gj, 
t  must  be  pre(s)  and  l'  must  be  post(s).  The  input  i  must  be  of  the  form  (i ,d,n).  Furthermore, 
T7]  =  Ti[x  1— >  n]  and  rf  =  rj[x  1— >  T],  This  means  that  72  has  the  form  (T2,  pre(s),r/). 

Let  q2  =  (r2[x  1— >  n],  post (s),rj[x  1— >  T]).  Since  Ti  =v  T2,  Ti[x  1— >  n]  r2[a:  t— >■  n\.  Thus, 

q'\  ~^2  and  d’2  ——►<72  in  3(model(s)). 

dt 

Case:  s  has  the  form  read(x’,  d)  with  d  =  df  or  d  =  dt.  In  this  case,  for  71  to  transition  to 
q[,  l  must  be  pre(s)  and  l'  must  be  post(s).  The  input  i  must  be  of  the  form  (\,d,  n).  Furthermore, 
r7]  =  Ti[x  e- >  n]  and  rf  =  r][x  1— >  F].  This  means  that  q2  has  the  form  (T2,  pre(s),rj). 

Let  q2  =  (r 2 [a?  1— >  n],  post(s),  rj[x  F]).  Since  Ti  =v  T2,  Ti[x  1— >  n\  =v  r2[x  1— >  n].  Thus,  q[  ~q2 

and  72  q'2  in  3  (model  (s)). 

dt 

Case:  s  has  the  form  print  (e,  d )  with  d  ^  dt.  In  this  case  q\  does  not  transition  to  any  other 
state  under  an  input  i  in  model  (s).  Thus,  the  statement  is  trivially  satisfied. 

Case:  s  has  the  form  print  (e,  dt)  •  Ditto. 

Case:  s  has  the  form  sa ;  Sf 

Since  Ls  is  the  disjoint  union  of  LSa,  LSb,  and  |pre(s),  post(s)},  one  of  the  following  cases  must 
hold: 


•  qi  and  q\  are  both  in  Qa.  statePolicy(model(s))  <  statePolicy(model(sa))  since  model(s)  has  at 
least  as  many  transitions  under  a  true  boolean  as  model  (sa).  The  needed  result  follows  from 
the  inductive  hypothesis. 

•  q\  and  q[  are  both  in  Q t,.  statePolicy(model(s))  <  statePolicy(model(sb))  since  model(s)  has 
at  least  as  many  transitions  under  a  true  boolean  as  model  (sb)-  •  The  needed  result  follows 
from  the  inductive  hypothesis. 

•  Qi  £  Qa  and  q[  £  Q b-  Since  q±  -3->g2  and  the  construction  of  model  (s),  there  must  exist  a 

d 

state  qia  of  the  form  (Tia,  post(sa),  rjia)  and  a  state  71b  of  the  form  (Tig,  pre(sb),  rjia)  such  that 

qi—^qu^qib-^q'i- 

dt  dt 

By  Lemma  25,  this  means  there  exists  a  state  q-2 a  such  that  g2 — >72a  and  gia~72a-  This 

dt 

implies  that  the  form  of  g2a  is  (r2a,  post(sa), rj\a).  Thus,  by  the  construction  of  model(s), 
<?2a^g2b  where  g2b  =  (r2a,  pre(sb),  ??ia). 

Since  gib  ~g2b>  the  inductive  hypothesis  on  Sb  applies  as  above  and  there  must  exist  q2  such 
that  g2b  — 72  and  q\  ~  q2 ■ 

dt 

•  qi  £  Qb  and  q[  £  Qa.  Since  gi  —-^q[  cannot  hold  in  this  case,  we  need  not  consider  it. 

dt 

•  gi  has  the  form  (Ti,  pre(s),  rj)  and  q\  is  in  Qa  or  Q b.  Since  gi~g2,  g2  must  have  the 
form  (r2,  pre(s),  rj)  where  Ti  ='n  F2.  By  the  construction  of  3model(s),  gi-»gia  where 
gia  =  (Ti,  pre(sa),??)  and  g2^72a  where  g2a  =  (r2,  pre(sa), 77).  Since  gia~72a  and  they  are 
both  in  3model(sa),  the  proof  continues  as  above. 

•  q\  has  the  form  (rj ,  post(s),  rf).  q±  —^q[  is  impossible  in  this  case,  so  we  need  not  consider 

dt 

it. 
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Case:  s  has  the  form  if  (e)saelsesb-  If  77(e)  =  T,  then  Ti(e)  =  ^(e)  since  Ti  =n  T2-  In  this 
case,  the  result  follows  from  using  the  inductive  hypothesis  on  sa  if  Ti(e)  7^  0  and  on  Sb  if  Ti(e)  =  0 
and  the  methods  used  above  for  dealing  with  the  cases  where  q\  has  the  form  (Id,  pre(s),r/}  or  q\ 
has  the  form  (T^,  post(s),  rj'). 

The  same  holds  even  if  77(e)  =  F  as  long  and  Ti(e)  =  T2(e). 

As  argued  in  Lemma  25,  cases  where  77(e)  =  F  and  either  sa  or  Sb  contains  a  while  loop, 
a  read  statement,  or  a  statement  of  the  form  print  (e,  dt)  are  handled  by  the  construction  of 

statePolicy(model(s)).  However,  if  no  read  statements  are  in  sa  or  Sb,  then  clearly  771  q\  cannot 

<k 

hold.  Thus,  all  cases  have  be  covered. 

Case:  s  has  the  form  while  (e)-sa  with  Ti(e)  7^  0.  If  77(e)  =  T,  then  Ti(e)  =  ^(e)  and  the 
inductive  hypothesis  may  be  applied  to  sa.  If  77(e)  =  F,  then  statePolicy(model(s))  would  allow 
information  to  flow  from  df  to  dt  at  q\.  As  above,  this  implies  that  df  dt  and  thus  the  result 
is  trivially  true. 

Case:  s  has  the  form  while  (e)sa  with  Ti(e)  =  0.  Ditto.  □ 

Now  to  prove  a  statement  slightly  stronger  than  Output  Consistency. 

Lemma  27.  For  all  statements  s  where  3(model(s))  =  (I,  O,  D,  dom,  Q,  770,  -*),  state-based  dynamic 
policies  q\,  q\ ,  7/2  €  Q,  and  o  €  O,  if  dom(o)  =  dt)  <statePol icy  ( model  (s)),  q\  ~  (72,  q\  — °-^  q[, 

dt 

and  df  9M1  dt.  then  there  must  exist  q'2  €  Q  such  that  q-2  —^q2  and  q[  ^  q2. 

dt  df 

Proof.  Since  q\r^q2  we  know  that  q\  has  the  form  (Ti,£,rj)  and  (T2,£,rj)  where  Ti  =n  T2-  Let 
Proof  by  induction  over  the  derivation  of  >s>. 

Case:  s  has  the  form  x:=e.  In  this  case  q±  does  not  transition  to  any  other  state  under  an 
input  o  in  model  (s).  Thus,  the  statement  is  trivially  satisfied. 

Case:  s  has  the  form  read(x,  d )  with  df  7^  d  7^  dt.  Ditto. 

Case:  s  has  the  form  read(x,  d)  with  d  =  df  or  d  =  dt.  Ditto. 

Case:  s  has  the  form  print  (e,  7 i)  with  d  7^  df.  In  this  case,  771  will  only  transition  to  another 
state  under  an  output  o  such  that  dom(7?)  7^  dt.  Thus,  the  statement  is  trivially  satisfied. 

Case:  s  has  the  form  print  (e,  df) .  q\  q[  if  77  =  (o,  dt,  Ti(e)),  L)  =  L 1 .  £  =  pre(s),  £'  = 
post(s),  and  7/  =  77. 

If  77(e)  =  T,  Ti(e)  =  T2(e)  since  Li  =q  T2.  Let  q2  =  (r2,  post(s),  77}.  q2^q2  and  q[  ~ q2. 

If  77(e)  =  F,  then  statePolicy(model(s))  would  allow  dt  access  to  df  at  77^.  The  fact  that  df  dt 

follows  from  the  fact  that  <statePolicy(model(s)).  Thus,  the  result  is  satisfied  trivially. 

The  remaining  cases  are  as  in  Lemma  26  just  replacing  — ^  with  — □ 

dt  dt 

Theorem  4  can  now  be  proved: 

Proof.  Lemmas  24,  26,  and  27  show  that  ~  is  an  unwinding  relation  for  statePolicy(model(s))  and 
3(model(s)).  Since  by  Lemma  23,  statePolicy(model(s))  is  a  non-revoking  safe  approximation  of 
policy(model(s))  this  means  that  3(model(s))  obeys  policy(model(s)).  By  Lemma  22,  this  means 
that  autom(s)  obeys  policy(model(s)).  □ 
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